Image binarization apparatus, image binarization method, image pickup apparatus, image pickup method, and a computer product

ABSTRACT

In a digital camera, a CPU divides a multi-valued image into blocks and selects object blocks, which are to be the object of processing. A CCD outputs luminance values of the multi-valued image. A low luminance threshold value setter sets low luminance threshold values on the basis of mean luminance values of blocks adjacent to the object block. A mean luminance value calculator calculates mean luminance values using luminance values from which luminance values that do not reach the low luminance threshold value in a block have been removed. A binarization threshold value setting circuit sets a binarization threshold value of the block based on the mean luminance values. A binarizer then binarizes the multi-valued image in the block on the basis of the binarization threshold values.

FIELD OF THE INVENTION

The present invention relates to an image binarization apparatus, animage pickup apparatus, an image binarization method, an image pickupmethod, and a computer product. More particularly, this inventionrelates to a technology an image is binarized after removing the shadowsand unevenness in the brightness of a multi-valued image input using animage input apparatus having an inconstant light source.

BACKGROUND OF THE INVENTION

Conventionally, when saving a document as an electronic documentgenerally a scanner, a camera, or the scanner section of a facsimilemachine is used as an image input apparatus. In this type of scanner(scanner section), a light source is provided inside the apparatus andlight emitted from the light source which is reflected from the documentis read by a CCD or the like. If necessary, the read image is thenbinarized and saved. Because the characteristics of the used lightsource and optical system are constant, shadows and unevenness generatedin the brightness are also constant and, therefore, can easily becorrected. Accordingly, high quality images can be output and digitalimages can be binarized easily and at high quality.

On the other hand, in recent years, with the rise in popularity of videocameras and digital cameras, there has been an increasing demand to beable to recognize written characters in images input from these types ofdevices. With regard to digital cameras, in particular, as the number ofpixels have increased markedly and their size reduced, they have begunto be used in various applications as portable information acquisitiontools. For example, character information such as documents, noticeboards, and advertisements provide sufficient information as binaryimages and, moreover, the amount of storage space required to save thisinformation is less than multi-valued images so that it is advantageousto save the information as a binarized image. Furthermore, binarizedimages can be transmitted using a facsimile or reused after undergoing acharacter recognition process.

Japanese Patent Application Laid-Open No. 3-237571 “Device ForCalculating Binarized Threshold of Image” discloses a technology ofsuitably binarizing these types of digital images. According to thistechnology a high quality binarized image is obtained by providing abinarization circuit which performs binarization processing on an imagein a window by comparing the difference between the brightness of eachpixel in the window and the brightness of a specific pixel with aparameter which is proportional to the contrast of the observed imageportion and by providing a determining circuit which determines thesuitability of the binary pattern obtained by the binarization circuitas an image pattern of a contour portion.

Japanese Patent Application Laid-Open No. 7-212591 “Image BinarizingDevice” also discloses a technology of binarization. According to thistechnology a histogram of luminance values is generated from amulti-valued image and the white pixel representative values and theblack pixel representative values determined from the histogram. Abinarization threshold value is then determined from the mean thereof,and the multi-valued image is binarized on the basis of the binarizationthreshold value. This patent application also discloses a technology inwhich an image is divided into blocks, the binarization threshold valueor white pixel representative value/black pixel representative value foreach block is determined, blocks with no characters therein areinterpolated from surrounding blocks, and block threshold values areadopted as the threshold values for each pixel, thereby allowingbrightness unevenness and shadows in the image to be removed. Thus, highquality binarization can be performed using the technology disclosed inthis patent application.

However, the problems described below exist in the conventionaltechnology. Namely, the number, position, and strength of the lightsources different, therefore, shadows and brightness unevenness easilyoccur in the photographed image. Moreover, because the shadows andbrightness unevenness are inconstant, it is not possible to applyuniform correction as with the scanner section of a photocopier machineor the like. Accordingly, images photographed using a digital camerahave the problem that high quality binarization is not possible.

Moreover, even if images have been photographed using a homogeneouslight source, peripheral light reduction is caused by the properties ofthe optical system resulting in the problem arising that uniformcorrection and high quality binarization are not achievable.

Furthermore, for example, on a notice board, some characters or a groupof characters are of small in size. If such a notice board isphotographed then there is a need to perform partial high qualitybinarization.

In Japanese Patent Application Laid-Open No. 3-237571 “Device ForCalculating Binarized Threshold of Image”, it is required to provide thebinarization circuit for performing image processing on each pixel in awindow, and the determination circuit for performing pattern matching onthe contour portions as an image pattern. Accordingly, there is theproblem that the processing load has substantial increased and theprocessing speed has consequently decreased.

Moreover, in Japanese Patent Application Laid-Open No. 7-212591 “ImageBinarizing Device”, in addition to generating a luminance valuehistogram for all the pixels, it is necessary to determine the white andblack pixel representative values using loop processing and perform edgeenhancement processing to enhance the edges of characters and diagrams.Accordingly, there is the problems that considerable processingresources are required and the rate of power consumption is also high.

SUMMARY OF THE INVENTION

It is one object of this invention to perform high quality binarizationon a multi-valued image.

It is an another object of the present invention to perform high qualitybinarization of a multi-valued image rapidly and at a low rate of powerconsumption.

In order to achieve the above object, the image binarization apparatusaccording to one aspect of the present invention removes the lowluminance values from an object block on the basis of mean luminancevalues of the surrounding blocks, and sets the binarization thresholdvalue of the object block on the basis of the mean of the luminancevalues from which the low luminance values have been removed.

The image binarization apparatus according to another aspect of thepresent invention removes the low luminance values on the basis of themean luminance values of surrounding blocks, sets the binarizationthreshold values of object blocks on the basis of the mean of theluminance values from which the low luminance values have been removed,and sets the binarization threshold values to be applied to each pixelof interpolation blocks on the basis of the binarization thresholdvalues of adjacent object blocks.

The image binarization apparatus according to still another aspect ofthe present invention removes the low luminance values of object blockson the basis of the mean luminance values of surrounding blocks, androunds the mean of the luminance values from which the low luminancevalues have been removed to values within a predetermined range. Thesevalues then become the basis on which the binarization threshold valuesof the object blocks are set.

The image binarization apparatus according to still another aspect ofthe present invention removes the low luminance values of object blockson the basis of the mean luminance values of surrounding blocks, androunds the mean of the luminance values from which the low luminancevalues have been removed to values within a predetermined range. Thesevalues then become the basis on which the binarization threshold valuesof the object blocks are set. The binarization threshold values appliedto each pixel of the interpolation blocks are set on the basis ofbinarization threshold values of adjacent object blocks.

The image pickup apparatus according to still another aspect of thepresent invention sets the binarization threshold values of createdblocks are set on the basis of smoothed photometric values of createdscreens.

The image pickup apparatus according to still another aspect of thepresent invention sets the binarization threshold values applied to eachpixel of an interpolation block on the basis of smoothed photometricvalues of adjacent created screens.

The image pickup apparatus according to still another aspect of thepresent invention rounds the photometric values of created screens tovalues within a predetermined range, and sets the binarization thresholdvalues of created blocks based on these values.

The image pickup apparatus according to still another aspect of thepresent invention rounds the photometric values of adjacent createdscreens to values within a predetermined range and sets the binarizationthreshold values applied to each pixel within an interpolation blockbased on these values.

The image binarization method according to still another aspect of thepresent invention removes the low luminance values of the object blockson the basis of the mean luminance values of surrounding blocks. Thebinarization threshold values of the object blocks are then set on thebasis of the mean of luminance values from which the low luminancevalues have been removed.

The image binarization method according to still another aspect of thepresent invention removes the low luminance values of the object blockson the basis of the mean luminance values of surrounding blocks. Thebinarization threshold values of the object blocks are then set on thebasis of the mean of the luminance values from which the low luminancevalues have been removed. Binarization threshold values to be applied toeach pixel of the interpolation block are then set based on thebinarization threshold values of adjacent object blocks.

The image binarization method according to still another aspect of thepresent invention removes the low luminance values on the basis of themean luminance values of surrounding blocks. The means of the luminancevalues from which the low luminance values have been removed are thenrounded to values within a predetermined range and the binarizationthreshold values of the object blocks are then set on the basis of thesevalues.

The image binarization method according to still another aspect of thepresent invention removes the low luminance values on the basis of themean luminance values of surrounding blocks. The means of the luminancevalues from which the low luminance values have been removed are thenrounded to values within a predetermined range and the binarizationthreshold values of the object blocks are then set on the basis of thesevalues. The binarization threshold values applied to each pixel of theinterpolation blocks are then set on the basis of the binarizationthreshold values of adjacent object blocks.

The image pickup method according to still another aspect of the presentinvention set the binarization threshold values of created blocks on thebasis of smoothed photometric values of a created screen.

The image pickup method according to still another aspect of the presentinvention sets the binarization threshold values applied to each pixelof an interpolation block on the basis of smoothed photometric values ofadjacent created screens.

The image pickup method according to still another aspect of the presentinvention rounds the photometric values of created screens to valueswithin a predetermined range, and sets the binarization threshold valuesof created blocks on the basis of these values.

The image pickup method according to still another aspect of the presentinvention rounds the photometric values of adjacent created screens tovalues within a predetermined range, and sets the binarization thresholdvalues applied to each pixel of the interpolation blocks on the basis ofthese values.

According to still another aspect of the present invention there isprovided a computer readable recording medium on which a program forenabling a computer to perform the functions of each step of the imagebinarization method and the image pickup method described above isrecorded.

Other objects and features of this invention will become apparent fromthe following description with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a structure of an imagebinarization apparatus of the first embodiment of the present inventionwhen it is used in a digital camera.

FIG. 2 is a conceptual view of a light receiving section of a CCD in adigital camera of the first embodiment.

FIG. 3A and FIG. 3B show examples of divisions of multi-valued imagesphotographed using a digital camera of the first embodiment into blocks.

FIG. 4A and FIG. 4B show examples of sampling intervals for samplingpixels inside a block.

FIG. 5 is a block diagram showing an example of a structure of a meanluminance value calculator of a digital camera of the first embodiment.

FIG. 6 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of thefirst embodiment.

FIG. 7 is a block diagram showing an example of the apparatus structurewhen an image binarization apparatus of the second embodiment of thepresent invention is used in a digital camera.

FIG. 8 is a block diagram showing an example of the apparatus structurewhen an image binarization apparatus of the third embodiment of thepresent invention is used in a digital camera.

FIG. 9A and FIG. 9B are explanatory diagrams summarizing the calculationof binarization threshold values applied to each pixel within aninterpolation block of a digital camera of the third embodiment.

FIG. 10 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of thethird embodiment.

FIG. 11 is a block diagram showing an example of the apparatus structureof a digital camera in which binarization threshold values applied toeach image pixel are calculated and image binarization is performed foreach pixel using a CMOS sensor in the image input section.

FIG. 12 is a block diagram showing an example of the apparatus structurewhen an image binarization apparatus of the fifth embodiment of thepresent invention is used in a digital camera.

FIG. 13 is a block diagram showing an example of the apparatus structurewhen an image binarization apparatus of the sixth embodiment of thepresent invention is used in a digital camera.

FIG. 14 is a block diagram showing an example of the image pickupapparatus of the seventh embodiment of the present invention when it isused in a digital camera.

FIG. 15 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of theseventh embodiment.

FIG. 16 is a block diagram showing an example of the apparatus structureas far as the binarization and recording of an input image in a digitalcamera using a CMOS sensor in the image input section.

FIG. 17 is a block diagram showing an example of the image pickupapparatus of the ninth embodiment of the present invention when it isused in a digital camera.

FIG. 18A and FIG. 18B show relationships between an interpolation blockand a screen in a digital camera of the ninth embodiment.

FIG. 19 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of theninth embodiment.

FIG. 20 is a structural diagram showing an example when an image pickupapparatus for calculating binarization threshold values applied to eachpixel in a predetermined block and for performing binarization of theimage data for each pixel using a CMOS sensor and a photometer is usedin a digital camera.

FIG. 21 is a block diagram showing an example of the apparatus structurewhen an image pickup apparatus of the eleventh embodiment of the presentinvention is used in a digital camera.

FIG. 22 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of theeleventh embodiment.

FIG. 23 is a block diagram showing another example of the apparatusstructure of a digital camera of the eleventh embodiment.

FIG. 24 is a block diagram showing an example of the apparatus structurewhen an image pickup apparatus of the twelfth embodiment of the presentinvention is used in a digital camera.

FIG. 25 is a flow chart showing the flow of image data processing as faras the binarization of a multi-valued image in a digital camera of thetwelfth embodiment.

FIG. 26 is a block diagram showing an example of the apparatus structurefrom an image input to a recording of a binarized image when an imagepickup apparatus, which performs image binarization in a CPU by softwareprocessing, is used in a digital camera.

FIG. 27 is a block diagram showing the apparatus structure from an imageinput to a recording of a binarized image when an image pickup apparatuswhich performs image binarization in a CPU by software processing isused in a digital camera.

FIG. 28 is a diagram showing an example of the structure of a computersystem when performing the present invention using software.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will now be described below in detail withreference to the drawings.

The first embodiment explained below relates to an image binarizationapparatus of the present invention which is used in a digital camera.FIG. 1 is a block diagram showing an example of the apparatus structurefrom the input of image data until a binarized image (binarized data) isrecorded when an image binarization apparatus of the present inventionis used in a digital camera.

A digital camera 100 comprises a CCD 101, an A/D converter 102, a whitebalance adjustor 103, a pixel interpolator 104, a luminance generator105, an aperture corrector 106, frame memory 107, a CPU 108, a blockbuffer 109, a mean luminance value calculator 120, a low luminancethreshold value setter 121, a binarization threshold value settingcircuit 122, a binarizer 123, a compressor 124, and an image storagememory 125.

The CCD 101 converts light converged by a not shown optical system ofthe digital camera 100 into electric signals and outputs R, G, B analogsignals for each pixel forming a multi-valued image as image data. Theoutput analog signals are converted into digital signals by the A/Dconverter 102. The white balance of the digital signals is adjusted bythe white balance adjustor 103. In the pixel interpolator 104,interpolation of R, G, or B signals for which no information exists isperformed in each pixel on the image data after the white balancethereof has been adjusted. Thereafter, R, G, and B are taken asrepresenting red, green, and blue, or alternatively, a red signal value,a green signal value, or a blue signal value.

The relationship between the filter of the CCD 101 and the interpolationby the pixel interpolator 104 will now be described. FIG. 2 is aconceptual view of a light receiving section of the CCD 101. R, G, and Bfilters are set in a fixed pattern in the light receiving section andcolor differences are identified by these filters. Note that it isnormal to provide more filters for green pixels to which the human eyeis highly sensitive than for other colors. The suffixes are used asidentifiers for identifying position (filter number) Note also thatfilter numbers are only provided for the central portion of the figure.

A red R interpolation signal value R(G0) and a blue B interpolationsignal value B(G0) at the position G0 are calculated using the followingequation (1):R(G 0)=(R 0+R 2)/2B(G 0)=(B 0+B 1)/2  (1)

Green G interpolation signal values G(R0), B(R0), G(B0), and R(B0) atthe positions R0 and B0 are calculated using the following equation (2):G(R 0)=(G 0+G 1+G 2+G 5)/4B(R 0)=(B 0+B 1+B 4+B 5)/4G(B 0)=(G 0+G 1+G 3+G 6)/4R(B 0)=(R 0+R 2+R 4+R 5)/4  (2)

The pixel interpolator 104 performs the interpolation expressed above ateach pixel position and outputs interpolated R, G, B signals for eachpixel.

The luminance generator 105 generates a luminance signal value Y fromeach of the interpolated pixels using the following equation (3):Y=0.34 R+0.55 G+0.11 B  (3)

A multiplier and an adder are necessary to calculate the luminancesignal values, which are digital values, using the equation (3).However, by approximating the equation (3) using the following equation(4), it is possible to construct the luminance generator 105 with onlyan adder.Y=(2/8)R+(5/8)G+(1/8)B  (4)

As a result, when this equation (4) is used to calculate the luminancesignal values, it is possible to calculate the luminance value Y with asimple circuit structure. Accordingly, it is possible to provide adigital camera with low circuit cost and superb calculation speed andpower consumption.

High pass portions of the image data of the luminance signal value Y areenhanced by the aperture corrector 106. The aperture correction isperformed using a common 5×5 size high pass enhancement filter. Afterhigh pass enhancement, the luminance value signals are storedtemporarily in the frame memory 107.

The CPU 108 calculates the block size and sampling interval (samplingcycle). The block buffer 109 described below, the mean luminance valuecalculator 120, and other circuits and members of the digital camera 100are also controlled by the CPU 108. Because the image size of themulti-valued image is determined by the size of the CCD (i.e. by thenumber of pixels), the CPU 108 calculates the necessary block size andsampling interval (sampling cycle) to binarize the multi-valued imagebased on the image size.

Unevenness in the luminance values caused by the optical system of thedigital camera 100 (for example, the lens) differ depending on theposition and strength of the light source, however, as a rule, thecenter region of an image is bright while the image tends to get darkerthe closer to the edges thereof. Accordingly, the CPU 108 calculates theimage division pattern taking into consideration peripheral lightreduction of the optical system. Note that a division pattern may beselected from a fixed block division pattern set in a not shown storagesection or the like.

FIG. 3A and FIG. 3B show examples of divisions of multi-valued imagesinto blocks. In addition to a normal division into rectangular shapes,FIG. 3A shows an example where a multi-valued image is divided into acombination of squares, rectangles, and triangles point-symmetricallyaround the center of the image (the lens center). In contrast, FIG. 3Bshows an example where a multi-valued image is divided on the basis ofconcentric circles around the center of the image. By dividing the imageinto blocks with due consideration given to the optical system, thebrightness of each block can be made more uniform. Moreover, becausebinarization threshold values are set for block units, as is describedbelow, high image quality binarization of a multi-valued image becomespossible. Note that, in the description given below, the image isdivided into square shaped blocks to make the description simpler.

The block buffer 109 reads from the frame memory 107 an image in theblock units of the block division pattern determined by the CPU 108 andtemporarily stores the image. Note that blocks stored in the blockbuffer 109 are referred to as object blocks (i.e. these blocks are theobject of subsequent processing).

The mean luminance value calculator 120 samples pixels from imagesstored in the block buffer 109 at a predetermined sampling cycle andcalculates the mean luminance value. FIG. 4A and FIG. 4B show examplesof sampling intervals at which the pixels in the blocks are sampled.FIG. 4A shows a state when the size of the image is 1280 pixels by 960pixels and the CPU 108 sets the block size at 64 pixels by 64 pixels andthe sampling cycle at 2 (FIG. 4A only shows a group of 9 pixels by 8pixels of a block). In contrast, FIG. 4B shows a state when the size ofthe image is 2560 pixels by 1920 pixels and the CPU 108 sets the blocksize at 128 pixels by 128 pixels and the sampling cycle at 4 (FIG. 4Bonly shows a group of 9 pixels by 8 pixels of a block).

In accordance with processing performance while considering powerconsumption, it is also possible for the CPU 108 to set as fixed a totalnumber of blocks for the size of the image to be binarized or a samplinginterval within a block. Accordingly, even if the size of the image islarger (i.e. even if the total number of pixels is greater), thesampling number can be fixed and the processing time until thebinarization threshold is determined can be shortened. Consequently,binarization processing with low power consumption is possible. Notethat the CPU 108 may set the sampling interval at units of one block.

FIG. 5 is a block diagram showing an example of the structure of themean luminance value calculator 120. The luminance value (here set at v)of the pixels sampled by the CPU 108 is compared with a low luminancethreshold value (here set at thl(i, j) (thl(i, j) is an indexrepresenting the block number and this block is set as B(i, j))) by acomparator 501. When the luminance value v of the sampled pixels isgreater than the low luminance threshold value thl(i, j), the comparator501 outputs a signal value of 1 and when the luminance value v of thesampled pixels is less than the low luminance threshold value thl(i, j),the comparator 501 outputs a signal value of 0.

If a signal value of 1 is output, a gate 507 is opened and the luminancevalue v is input into the adder 506. The adder 506 adds a value of anaddition result register 502 (referred to hereafter as sumv) to theinput luminance value v of the pixels and the addition result register502 stores the new addition result. The signal value of 1 from thecomparator 501 is also transmitted to a counter 503 which counts thenumber of luminance values v (referred to hereafter as num) to havepassed through the gate 507.

The above processing can be expressed as an algorithm in a mathematicalequation by the following equation (5).if v>thl(i, j)then sumv=sumv+vnum=num+1else sumv=sumvnum=num  (5)

If, as a result of the increment, the counter 503 is placed in a statewhere the number of digits is steadily increasing (i.e. in a state wherethe counter expresses a power of two), the gate 508 is opened and thesum of the luminance values v (sumv) held in the addition resultregister 502 is transmitted to a shift register 504. A bit positionbeginning with a 1 in the counter 503 is shifted to the right by anamount of −1. After processing has been completed for all pixels in ablock, the value stored in the shift register 504 is output as a meanluminance value ave(i, j). Namely, the mean luminance value iscalculated with the following equation (6):ave(i, j)=sumv′/num′  (6)

Here, num′ is a numerical value less than the sample number sampled inthe block B(i, j) and represents the greatest value of a numberexpressed as a power of two. While sumv′ represents the value held inthe addition result register 502 when the num′ is counted.

The low luminance threshold value setter 121 multiplies a predeterminedcoefficient by the mean luminance value ave(i−1, j) of the previousadjacent block (i.e. if the current block is B(i, j), then the previousadjacent block is B(i−1, j)) and calculates the low luminance thresholdvalue thl(i, j) to be used in the mean luminance value calculator 120.If the predetermined coefficient is taken as Ca=¼ (1/the power of 2),then the low luminance value threshold setter 121 becomes a value fromwhich lower order 2 bits of the ave(i, j) are removed and, consequently,no special circuitry is required and the circuit structure can besimplified. Moreover, high speed processing at a low level of powerconsumption becomes possible. The calculation described above can berepresented by the following equation (7):thl(i, j)=ave(i−1, j)×Ca  (7)

In this case, the mean luminance value ave of only one adjacent blockwas used when the low luminance threshold value thl was calculated,however, depending on the type of block division, it is also possible touse the mean luminance values ave of all adjacent blocks (for example,those blocks above, below, to the left and to the right).

The binarization threshold value setting circuit 122 uses the meanluminance value ave(i, j) calculated by the mean luminance valuecalculator 120 to set a binarization threshold value TH(i, j) to be usedfor binarizing a multi-valued image. The binarization threshold valueTH(i, j) is set by multiplying the mean luminance value ave(i, j) by,for example, a predetermined coefficient Cb. However, in the same way aswith the low luminance threshold value setting circuit 121, if Cb is setas x/16 or Cb is set as x/8 (where x is a predefined value representinga natural number no greater than the denominator), then the binarizationthreshold value setting circuit 122 can be structured using only anadder. Accordingly, the circuit structure can be simplified and highspeed processing at a low level of power consumption becomes possible.If the above is represented as a mathematical equation, the followingequation (8) is obtained:TH(i, j)=ave(i, j)×Cb  (8)

The binarizer 123 compares each pixel of the block buffer 109 with thebinarization threshold value TH and performs binarization. The binarizedimage undergoes image compression appropriate to a binary image such asMH or MR in the compressor 124. The compressed image is then stored inthe image storage memory 125.

In the above description, the CPU 108 corresponds to the block divisionunit and sampling unit; the CCD 101, the A/D converter 102, the whitebalance adjuster 103, the pixel interpolator 104, the luminancegenerator 105, and the aperture corrector 106 correspond to theluminance value output unit; the binarization threshold value settingcircuit 122 corresponds to the binarization threshold value settingunit; the binarizer 123 corresponds to the binarization unit; the lowluminance threshold value setter 121 corresponds to the low luminancethreshold value setting unit; the frame memory 107, the block buffer109, and the CPU 108 correspond to the object block selection unit; thecomparator 501 corresponds to the low luminance value removal unit, andthe mean luminance value calculator 120 corresponds to the meanluminance value calculation unit.

Next, the flow of the processing as far as the binarization of amulti-valued image is described. FIG. 6 is a flow chart showing the flowof image data processing until a multi-valued image is binarized. TheCPU 108 reads the size of a multi-valued image stored in the framememory 107 (step S601). Note that it is also possible to use informationfrom a preset number of pixels in the digital camera 100 or from the CCD101 or the like, instead of from the frame memory 107. Next, inaccordance with the image size, the CPU 108 sets the sampling cycle(step S602).

On the basis of the image size read in step S601 and the sampling cycleset in step S602, the CPU 108 sets the block size, shape, and divisionpattern (step S603). Because the number of pixels output by the CCD 101remains fixed or is a rated pixel number such as 640×480 pixels or800×600 pixels specified by switching mode, it is also possible for theCPU 108 to select a predetermined sampling cycle and block shape.

Next, a single block B(i, j) is set from the plurality of created blocksand image information is transferred to the block buffer 109 (stepS604). Pixels (luminance values) are then sampled from the block B(i, j)in accordance with the sampling cycle set in step S602 (step S605).Next, on the basis of the mean luminance value ave(i−1, j) of theadjacent block B(i−1, j) calculated in a previous routine, a lowluminance threshold value thl(i, j) for the block B(i, j) is calculatedand luminance values which do not reach the low luminance thresholdvalue thl(i, j) are then removed from luminance values of the sampledpixels (step S606). Removing the low luminance pixels in step S606 aidshigh quality binarization processing.

Using the luminance values from which the low luminance values have beenremoved, the mean luminance value ave(i, j) of the block B(i, j) iscalculated (step S607). The binarization threshold value TH(i, j) forthe block B(i, j) is then calculated on the basis of the mean luminancevalue ave(i, j) (step S608). The binarization threshold value TH(i, j)is then used to binarize all the pixels (taken as g(x, y)) of the blockB(i, j) stored in the block buffer 109 (step S609). Note that x and yare natural numbers representing the position of each pixel inside theblock.

Lastly, a determination is made as to whether or not the binarizationprocessing has been completed for all blocks (step S610). When all theblocks have been binarized (i.e. the result of determination in stepS610 is YES), the processing is ended. If not all the blocks have beenbinarized (i.e. the result of determination instep S610 is NO), a blockadjacent to the block B(i, j) (for example, the block B(i+1, j)) is setand steps S604 to S610 are repeated.

In the first embodiment, an example was described when the imagebinarization apparatus of the present invention was used in a digitalcamera. Block division is performed to correspond to the image size andthe optical system and samples of the luminance values are extractedfrom the blocks. Because of this and because a low luminancebinarization threshold value is set in consideration of the surroundingblocks and binarization is performed based on this, even if the all thepixels inside a block are part of a large character, or if thebrightness between blocks abruptly changes, it is possible to calculatean appropriate mean luminance value and to perform high quality, rapidbinarization of an image at a low level of power consumption. This, inturn, enables the useable duration of the digital camera battery to belengthened.

The second embodiment explained below relates to an image binarizationapparatus of the present invention which is used in a digital camera.FIG. 7 is a block diagram showing an example of the apparatus structurefrom image input until a binarized image is recorded in a digital camerausing a CMOS sensor in the image input section. Note that, in thepresent embodiment, structural sections identical to those in the firstembodiment are given the same symbols and a detailed description thereofis omitted. Here, mainly those sections different to those of the firstembodiment are described.

The digital camera 700 is provided with a CMOS sensor 701 in the imageinput section. As a result, unlike the CCD 101 (see FIG. 1) which canonly perform raster scanning, the CMOS sensor 701 is capable of randomaccess and cam perform reading in block units. Therefore, the framememory 107 is not required and the circuit structure can be simplified.Furthermore, unlike the CCD 101 which requires a separate power sourceto the CMOS integrated circuits which form the other circuits of thedigital camera 700, the CMOS sensor 701 uses the same power source asthe CMOS integrated circuits, thus reducing power consumption.Consequently, because the size of the circuitry making up the digitalcamera 700 is also reduced, it is far more convenient than a systemusing a CCD in terms of power consumption, processing speed, and cost.Note that a CMOS sensor is used in the present embodiment, however,another image binarization apparatus having a block accessible imageinput section may also be used.

In this third embodiment, a description is given of when a imagebinarization apparatus which calculates binarization threshold values tobe applied to each pixel in a predetermined block and binarizes theimage data for each pixel is used in a digital camera. FIG. 8 is a blockdiagram showing an example of the apparatus structure of a digitalcamera which calculates binarization threshold values to be applied toeach pixel and binarizes the image data for each pixel. Note that, inthe present embodiment, structural sections identical to those in thefirst embodiment are given the same symbols and a detailed descriptionthereof is omitted. Here, mainly those sections different to those ofthe first embodiment are described.

The digital camera 800 is provided with a block binarization thresholdvalue setting circuit 801 into which are input mean luminance valuesoutput from the mean luminance value calculator 120 and which outputsbinarization threshold values to be applied to each block (these arereferred to below as block binarization threshold values); memory 802for storing the block binarization threshold values; and a binarizationthreshold value interpolator 803 which sets binarization thresholdvalues to be applied to each individual pixel in a predetermined blockon the basis of the block binarization threshold values. Other thanthese, the structure is the same as that of the digital camera 100 ofthe first embodiment. Note that a section for generating colordifference signals is omitted.

The block binarization threshold value setting circuit 801 is identicalto the binarization threshold value setting circuit 122 of the digitalcamera 100 (see FIG. 1) and outputs block binarization threshold valuesof object blocks for storage in the memory 802. The block binarizationthreshold value setting circuit 801 calculates block binarizationthreshold values for all the blocks to be stored sequentially in thememory 802.

The binarization threshold value interpolator 803 uses the blockbinarization threshold values of all the blocks stored in the memory 802to set binarization threshold values to be applied to each pixel in apredetermined area. Note that, in the description given below, the terminterpolation block refers to a predetermined area for each of whoseconstituent pixels a binarization threshold value is set. A descriptionwill now be given of the outline of the calculation of the binarizationthreshold values to be applied to each pixel in an interpolation block.FIG. 9A and FIG. 9B are explanatory diagrams summarizing the calculationof binarization threshold values applied to each pixel within aninterpolation block. FIG. 9A shows the relationship between objectblocks and an interpolation block, and FIG. 9B shows the calculation ofthe binarization threshold value to be applied to each pixel within aninterpolation block.

As is clearly shown in FIG. 9A, the interpolation block BH bridges fouradjacent object blocks Ba, Bb, Bc, and Bd. The binarization thresholdvalues of the object blocks Ba, Bb, Bc, and Bd are set respectively asa, b, c, and d. The binarization threshold value interpolator 803 usesthe block binarization threshold values a, b, c, and d to calculatebinarization threshold values to be applied to each pixel within theinterpolation block BH.

A description will now be given of the method of calculatingbinarization threshold values to be applied to each pixel bp inside theinterpolation block BH, while referring to FIG. 9B. The shape of theinterpolation block BH is taken as being a rectangle with the size(number of pixels) thereof set as xbnum in the horizontal direction andybnum in the vertical direction. The position of the pixel bp is set as(m, 1). At this time, it may be considered that the value a is at thepoint (0, 0) of the interpolation block BH, the value b is at the point(xbnum, 0) of the interpolation block BH, the value c is at the point(0, ybnum) of the interpolation block BH, and the value d is at thepoint (xbnum, ybnum) of the interpolation block BH. The area betweenthese points is linearly approximated.

Firstly, if the temporary threshold value on the boundary line at theleft hand side of the interpolation block BH is taken as leftth, and thetemporary threshold value on the boundary line at the right hand side ofthe interpolation block BH is taken as rightth, then the followingequation (9) can be applied:leftth=(a(ybnum−1)+c1)/ybnumrightth=(b(ybnum−1)+d1)/ybnum  (9)

Next, if it is taken that the point (0, 1) has the value leftth and thepoint (xbnum, 1) has the value rightth, then the binarization thresholdvalue th(m, 1) to be applied to the pixels bp is linearly approximatedusing the following equation (10):th(m, 1)=(leftth(xbnum−m)+rightth×m)/xbnum  (10)

Note that when the interpolation block BH is at an edge of the overallimage, there are only one or two adjacent object blocks. In this case,by substituting the obtained block binarization threshold values for theunavailable block binarization threshold values, equations (9) and (10)can be used. For example, at the top left edge of the image the onlyobtainable block binarization threshold value is d and the blockbinarization threshold values a, b, and c are not obtainable.Accordingly, in this case, the value d is substituted for a, b, and c.

In the binarizer 123, the luminance values of the pixels bp in theinterpolation block obtained from the frame memory 107 or the blockbuffer 109 are compared with the binarization threshold values for thepixels bp calculated by the binarization threshold value interpolator803 and binarization of the luminance values of the interpolation blockBH is performed. Image compression appropriate for a binarized imagesuch as MH or MMR is performed on the binarized image data using thecompressor 124 in the same way as in the first embodiment.

Next, a description will be given of the processing flow until amulti-valued image is binarized in the present embodiment. FIG. 10 is aflow chart showing the flow of image data processing until themulti-valued image is binarized. The CPU 108 reads the size of themulti-valued image stored in the frame memory 107 (step S1001). Notethat it is possible to use information from the CCD 101 or a pixelnumber preset in the digital camera 100 or the like instead of from theframe memory 107. Next, in accordance with the image size, the CPU 108sets the sampling cycle (step S1002).

On the basis of the image size read in step S1001 and the sampling cycleset in step S1002, the CPU 108 sets the block size, shape, and divisionpattern (step S1003). Because the number of pixels output by the CCD 101remains fixed or is a rated pixel number such as 640×480 pixels or800×600 pixels specified by switching mode, it is also possible for theCPU 108 to select a predetermined sampling cycle and block shape.

Next, a single block B(i, j) is set from the plurality of blocks createdand image information is transferred to the block buffer 109 (stepS1004). Pixels (luminance values) are then sampled from the block B(i,j) in accordance with the sampling cycle set in step S1002 (step S1005).Next, on the basis of the mean luminance value ave(i−1, j) of theadjacent block B(i−1, j) calculated in a previous routine, the luminancethreshold value thl(i, j) for the block B(i, j) is calculated. Luminancevalues which do not reach the low luminance threshold value thl(i, j)are then removed from luminance values of the sampled pixels (stepS1006). Removing the low luminance pixels in step S1006 aids highquality binarization processing.

Using the luminance values from which the low luminance values have beenremoved, the mean luminance value ave(i, j) of the block B(i, j) iscalculated (step S1007). The block binarization threshold value TH(i, j)for the block B(i, j) is then calculated on the basis of the meanluminance value ave (i, j) (step S1008). The calculated blockbinarization threshold value TH is then stored in the memory 802 (stepS1009) A determination is then made as to whether or not the blockbinarization threshold values have been calculated for all blocks (stepS1010). If the calculations have not been completed (i.e. if the resultof determination in step S1010 is NO), a block adjacent to the blockB(i, j) (for example, the block B(i+1, j)) is set and steps S1004 toS1010 are repeated.

When block binarization threshold values have been calculated for allthe blocks (i.e. the result of determination instep S1010 is YES), aninterpolation block is set (step S1011). This interpolation block issuitable for performing high quality partial binarization and the likeand may be set in advance by the user or may be set by a suitable modeswitching so as to comprise the central portion of the image. Thebinarization threshold value interpolator 803 uses the blockbinarization threshold values TH of the object blocks bridged by theinterpolation block set in step S1011 to set the binarization thresholdvalues th(x, y) of each single pixel inside the interpolation block(step S1012). Note that the coefficient f in FIG. 10 conceptuallyrepresents equations (9) and (10), while x and y are natural numbersrepresenting the positions of each pixel in an image.

The pixels (taken here as g(x, y)) read from the frame memory 107 arebinarized by the binarizer 123 using the binarization threshold valuescalculated in step S1011 (step S1013). Next, a determination is made asto whether or not binarization processing has been performed on all thepixels (step S1014). If binarization processing has not been performedon all the pixels (i.e. if the result of determination in step S1014 isNO), then the pixel adjacent to the pixel g (x, y) (for example, thepixel g(x+1, y)) is set and the steps S1012 to S1014 are repeated. Ifthe determination is made that binarization processing has beencompleted for all the pixels (i.e. if the result of determination instep S1014 is YES), a determination is made as to whether or notbinarization processing has been completed for all interpolation blocks(step S1015). If the binarization processing has not been completed (ifthe result of determination in step S1015 is NO), then steps S1011 toS1015 are repeated. When the binarization processing has been completed(when the result of determination in step S1015 is YES), the processingis ended.

In the third embodiment, block division is performed to correspond tothe image size and the optical system and samples of the luminancevalues are extracted from the blocks. Because of this and because a lowluminance binarization threshold value is set in consideration of thesurrounding blocks and binarization is performed based on this, and, inaddition, because binarization is performed by setting binarizationthreshold values to be applied to each pixel in an interpolation blockusing the above block binarization threshold values, high quality imageprocessing using the digital camera of the first embodiment becomespossible. In particular, when photographing a notice board or the like,when all of the characters are small or when characters in a portion ofthe area are small, it is possible to perform partial high qualitybinarization in those locations.

The third embodiment explained below relates to an image binarizationapparatus of the present invention which uses a CMOS sensor whencalculating binarization threshold values to be applied to each pixel ina predetermined block and performing binarization of the image data foreach pixel is used in a digital camera. FIG. 11 is a block diagramshowing an example of the apparatus structure of a digital camera whichuses a CMOS sensor in the image input section, calculates binarizationthreshold values to be applied to each pixel, and performs binarizationof the image data for each pixel. Note that, in the present embodiment,structural sections identical to those in the third embodiment are giventhe same symbols and a detailed description thereof is omitted. Here,mainly those sections different to those of the third embodiment aredescribed.

The digital camera 1100 is provided with a CMOS sensor 1101 in the imageinput section. As a result, unlike the CCD 101 (see FIG. 8) which canonly perform raster scanning, the CMOS sensor 1101 is capable of randomaccess, reading a block as a single unit, reading pixel values for eachblock, calculating block binarization threshold values, and storing datain the memory 802. Therefore, the block buffer 109 (see FIG. 8) is notrequired and the circuit structure can be simplified.

At the same time, the pixel values read per block are converted toluminance signals, undergo aperture correction, and are stored in theframe memory 107. Furthermore, unlike the CCD 101 which requires aseparate power source to the CMOS integrated circuits which form theother circuits of the digital camera 800, the CMOS sensor 1101 uses thesame power source as the CMOS integrated circuits, thus reducing powerconsumption. Consequently, because the size of the circuitry making upthe digital camera 1100 is also reduced, it is far more convenient thana system using a CCD in terms of power consumption, processing speed,and cost. Note that a CMOS sensor is used in the present embodiment,however, another image binarization apparatus having a block accessibleimage input section may also be used.

In the fifth embodiment, a description is given of when an imagebinarization apparatus in which the spread of the mean luminance valuesis limited so that the values are contained within a predetermined rangeis used in a digital camera. FIG. 12 is a block diagram showing anexample of the apparatus structure of a digital camera in which thespread of the mean luminance values is limited so that the values arecontained within a predetermined range. Note that structural sections ofthe present embodiment identical to those in the first embodiment aregiven the same symbols and a description thereof is omitted.

The digital camera 1200 is provided with a limiter 1201 which limits themean luminance values output from the mean luminance value calculator120 using a preset lower limit value, so that mean luminance valueswhich are too low are prevented from being outputted. An example thereofmight be relatively bright characters in a dark image, or a morespecific example might be a picture of a cat drawn in white chalk on ablackboard. Compared to the area of the blackboard, the area of thewhite chalk figure is extremely small, however, in this case, theimportant information is the image of the cat drawn in white chalk. Ifnormal binarization is performed in this case, the image data of theportions of the blackboard other than those portions containing theimage of the cat in white chalk becomes dominant. As a result, thebinarization threshold value ends up being set at the boundaries betweenlight and dark of the black portions of the blackboard. Consequently,the white chalk image is naturally judged to be white, however, whitechalk dust and fragments (for example, the track left by a blackboarderaser or the like) are also judged to be white and the end result isthat noise is increased.

The limiter 1201 adjusts the mean average luminance values so that thistype of noise is not reproduced. Specifically, a determination is madeas to whether or not any mean luminance values exist within apredetermined range and, if the result of this determination isnegative, then the mean value is substituted for the lower limit valueor upper limit value of that range. By setting the lower limit (or upperlimit) of the mean luminance value using the limiter 1201, it ispossible to obtain a binarized image with high contrast. Depending onthe mode of use, it is also possible to limit the mean luminance valueby a predetermined upper limit value so that a mean luminance valuewhich is too high is not output.

Note that, in the present embodiment, a CCD 101 is used, however,depending on the mode of use, it is also possible to use a CMOS sensor701 (see FIG. 7) as in the second embodiment.

In this sixth embodiment, the description given is of when a digitalcamera uses an image binarization apparatus in which binarizationthreshold values applied to each pixel in a predetermined block arecalculated and binarization of the image data performed for each pixel,with the spread of the mean luminance values limited so as to be withina predetermined range. FIG. 13 is a block diagram showing an example ofthe structure of a digital camera according to the present embodiment.Note that portions of the structure of the present embodiment which areidentical to those of the fifth embodiment are given the same symbolsand a description thereof is omitted.

The digital camera 1300 is provided with a limiter 1301 which limits themean luminance values output from the mean luminance value calculator120 using a preset upper limit value, so that mean luminance valueswhich are too low are prevented from being outputted. Mean luminancevalues output from the limiter 1301 are within a predetermined range.For example, mean luminance values which stand out due to being affectedby the reflected flash from a whiteboard or the like are adjusted so asto fall within a predetermined range. Accordingly, block binarizationthreshold values set by the block binarization threshold value settingcircuit 801 can be prevented from being inconsistent with the values ofother blocks.

In particular, in the present embodiment, binarization threshold valuesto be applied to each pixel in an interpolation block are calculated bythe binarization threshold value interpolator 803 using the blockbinarization threshold values of surrounding object blocks. At thistime, because mean luminance values which standout are removed by thelimiter 1301, each binarization threshold value can be moreappropriately set. As a result, even higher quality binarizationprocessing becomes possible. Depending on the mode of use, it is alsopossible for the mean luminance values to be limited by a lower limitvalue set in advance so that mean luminance values which are too low arenot output. Moreover, the limiter 1301 may also set a predeterminedrange using the block binarization threshold values.

Note that, in the present embodiment, a CCD 101 is used, however,depending on the mode of use, it is also possible to use a CMOS sensor1101 (see FIG. 11) as in the fourth embodiment.

The seventh embodiment explained below relates to an image binarizationapparatus having a photometry unit. FIG. 14 is a block diagram showingan example of the apparatus structure from an image input until arecording of a binarized image when an image pickup apparatus providedwith a photometer is used in a digital camera. Note that, in the presentembodiment, because the structural elements are similar to those of thefirst embodiment, the same structural elements as those of the firstembodiment are given the same symbols and a detailed description thereofis omitted. Those portions that are different to the first embodimentare mainly described.

The digital camera 1400 comprises the CCD 101, the A/D converter 102,the white balance adjustor 103, the pixel interpolator 104, theluminance generator 105, the aperture corrector 106, the frame memory107, the CPU 108, a photometer 1401, a smoother 1402, memory 1403, ablock reading controller 1404, the binarization threshold value settingcircuit 122, the binarizer 123, the compressor 124, and the imagestorage memory 125.

The CCD 101 converts light converged by a not shown optical system ofthe digital camera 1400 into electric signals and outputs R, G, B analogsignals for each pixel forming a multi-valued image. The output analogsignals are converted into digital signals by the A/D converter 102. Asthey pass through the white balance adjustor 103, the pixel interpolator104, the luminance generator 105, and the aperture corrector 106, thedigital signals undergo processing such as luminance valueinterpolation, extraction, and the like and are then temporarily storedin the frame memory 107.

Based on luminance information from the photometer 1401 (describedbelow), the CPU 108 controls the block reading controller 1404(described below) and divides an image stored in the frame memory 107 inthe way same as the screen division used by the photometer 1401 forphotometry. The screen division used for photometry by the photometer1401 may also be fixed. Alternatively, it may also be divided throughthe control of the CPU 108, in the same way as in FIG. 3. The CPU 108controls the other circuits and members of the digital camera 1400.

The photometer 1401 is provided with an automatic exposure (AE)detection mechanism for performing photometry on an object to bephotographed before photographing an image, and measures the brightnessof each screen based on digital signals output from the A/D converter102. The method of the photometry may, for example, comprise measurementby adding the luminance values of the pixels. At this time, it is alsopossible for the CPU 108 to sample photometric values for the photometer1401 to use in the addition. Note that, depending on the mode of use, itis also possible to use a light amount photo-electrically converted bythe CCD 101.

The smoother 1402 smoothes the photometric values of each screenobtained by the photometer 1401 and outputs them as ave(i, j) to thememory 1403. The processing below is an example of this smoothing. Whenthe mean value ave(i, j) of the photometric values of all the pixelscontained in one screen (this screen is referred to here as G(i, j))within the photometer 1401 (alternatively, the mean value of thephotometric values sampled from within the screen G(i, j)) isinconsistent with the mean values of the photometric values of thesurrounding screens, the photometric values of each pixel of the screenG(i, j) are corrected so as to be consistent with the mean values of thephotometric values of the surrounding screens.

An example of an algorithm for performing this processing is givenbelow. The mean photometric value of the four screens adjacent to thescreen G(i, j) is set as ave4(i, j). If the mean photometric valueave(i, j) of the screen G(i, j) is three times the value ave4(i, j) ormore, then the photometric values of each pixel in the screen G(i, j)(referred to here as s(x, y)) are converted using the following equation(11): $\begin{matrix}{{{{if}\quad{{ave}\left( {i,j} \right)}} \geq {3 \times {{ave4}\left( {i,j} \right)}}}{{{then}\quad{s\left( {i,j} \right)}} = {{{ave4}\left( {i,j} \right)} + {\left( {1/4} \right) \times \left( {{s\left( {i,j} \right)} - {{ave4}\left( {i,j} \right)}} \right)}}}} & (11)\end{matrix}$

In the smoother 1402, the converted photometric values are used torecalculate the mean photometric value ave(i, j) of the screen G(i, j)and the mean photometric value is then outputted to the memory 1403. Thememory 1403 stores this mean photometric value.

The block reading controller 1404 divides the images stored in the framememory 107 in the same way as the screen division used for photometry bythe photometer 1401. As a result, the binarization threshold values aremade more natural.

The binarization threshold value setting circuit 122 sets thebinarization value TH(i, j) based on the ave(i, j). The setting unit isa multiplier for multiplying the ave(i, j) by a preset coefficient Cb,however, if the coefficient Cb is made to equal x/16 or Cb is made toequal x/8 (wherein x is a predetermined value representing a naturalnumber no greater than the denominator), then the binarization thresholdvalue setting circuit 122 can be constructed simply from an adder, whichis efficient in terms of both cost and speed.

Based on the binarization threshold value TH(i, j), the binarizer 123binarizes each pixel of the block B(i, j) corresponding to the screenG(i, j). Note that smoothed blocks are binarized through appropriateprocessing using converted photometric values. The compressor 124performs compression appropriate to a binary image such as MH or MR. Thecompressed image is then stored in the image storage memory 125.

In the above description, the CCD 101 corresponds to the image pickupunit; the CPU 108 and the photometer 1401 correspond to the screendivision unit; the photometer 1401 corresponds to the photometry unit;the CPU 108 corresponds to the block division unit; the binarizationthreshold value setting circuit 122 corresponds to the binarizationthreshold value setting unit; the binarizer 123 corresponds to thebinarization unit; and the smoother 1402 corresponds to the photometricvalue smoothing unit.

Next the flow of the image data processing as far as the binarization ofa multi-valued image will be described. FIG. 15 is a flow chart showingthe flow of data processing as far as the binarization of multi-valuedimage. Firstly, the CPU 108 reads the size of the image from the CCD 101(step S1501). Next, the CPU 108 sets a screen G(i, j) from thephotometric values output from the photometer 1401 (step S1502) andcalculates the mean value of the photometric values of that screen (stepS1503). The mean may be determined using all the photometric values orusing an appropriate sampling thereof.

Next, a determination is made as to whether or not the mean value of thephotometric values of the screen is inconsistent with the mean values ofthe photometric values of adjacent screens (step S1504). If the meanvalue is inconsistent (i.e. the result of determination in step S1504 isYES), then the mean value of the photometric values is smoothed usingthe smoother 1402 (step S1505). If the mean value is not inconsistent(i.e. the result of determination in step S1504 is NO), oralternatively, if the smoothing processing of step S1505 has beencompleted, the block reading controller 1404 reads the block B(i, j)which corresponds to the screen G(i, j) from the frame memory 107 (stepS1506) and calculates the binarization threshold value based on thephotometric value of the screen G(i, j) (Step S1507). The multi-valuedimage within that block is then binarized using this binarizationthreshold value (step S1508).

A determination is then made as to whether or not binarization of allblocks has been completed (step S1509). If all blocks have beenbinarized (i.e. if the result of determination in step S1509 is YES),the binarization processing is completed. If not all blocks have beenbinarized (i.e. if the result of determination in step S1509 is NO), theroutine returns to step S1502, the next screen (for example, the screenG(i+1, j)) is set, and the subsequent steps S1502 to S1509 are repeated.

In this seventh embodiment, because a binarization threshold value isset using information (photometric values) obtained from an automaticphotometry section provided in a digital camera, there is no need forseparate processing to set the binarization threshold value. As aresult, the circuit structure is simplified and the cost thereofreduced. At the same time, because separate binarization threshold valuecalculation processing is unnecessary, power consumption can be reduced.In addition, even if the image has pinpoint reflections from a lightsource, high quality binarization of a multi-valued image is possible.

The eighth embodiment explained below relates to an image pickupapparatus using a CMOS sensor is described. FIG. 16 is a block diagramshowing an example of the apparatus structure until an input image isbinarized and recorded of a digital camera using a CMOS sensor in theimage input section thereof. Note that, in the present embodiment,portions identical to those of the seventh embodiment are given the samedescriptive symbols as in the seventh embodiment and a descriptionthereof is omitted. The description given here is of those portions thatdiffer from the seventh embodiment.

The digital camera 1600 is provided with a CMOS sensor 1601 in the imageinput section thereof. Accordingly, unlike the CCD 101 which can onlyperform raster scanning (see FIG. 14), the CMOS sensor 1601 is capableof random access and reading single block units. Therefore, the framememory 107 and the block reading controller 1404 are unnecessaryallowing the circuit structure to be simplified. In this case, the CPU108 performs the functions of the block reading controller.

Further, in contrast, to the CCD 101 which requires a separate powersource to the CMOS integrated circuit, the CMOS sensor 1601 is able touse the same power source as the CMOS integrated circuit, enabling powerconsumption to be reduced. As a result, because the scale of thecircuitry of the digital camera 1600 can also be reduced, this system isfar more efficient in terms of power consumption, processing speed andcost than a system using a CCD. Note that, in the present embodiment, aCMOS sensor was employed, however, any other image pickup apparatushaving a block accessible image input section may be used.

The ninth embodiment explained below relates to an image pickupapparatus which is provided with a photometry unit, which calculatesbinarization threshold values to be applied to each pixel in apredetermined block, and which performs binarization of image data foreach pixel is used in a digital camera. FIG. 17 view shows an example ofthe structure when an image pickup apparatus which is provided with aphotometer and which sets a binarization threshold value for each pixelis used in a digital camera. Note that, because structural elements ofthe present embodiment are similar to those of the seventh embodiment,those structural elements which are identical to those of the seventhembodiment are given the same descriptive symbols and a descriptionthereof is omitted. The description given here is of those portions thatdiffer from the seventh embodiment.

The digital camera 1700 is provided with a screen binarization thresholdvalue setting circuit 1701 into which are input mean values ofphotometric values output from the smoother 1402 and stored in thememory 1403, and for outputting values multiplied by a predeterminedcoefficient (referred to below as screen binarization threshold values)for each screen; memory 1702 for storing screen binarization thresholdvalues; and a binarization threshold value interpolator 1703 for settingbinarization threshold values to be applied to each individual pixel ina predetermined block based on the screen binarization threshold values.Note that a section for generating color difference signals is omitted.

The screen binarization threshold value setting circuit 1701 multipliesa coefficient corresponding to a particular screen by a smoothedphotometric value. In order to simplify the description below, thescreen having the smoothed photometric value a is referred to as Ga,while the predetermined coefficient is referred to as Cb(Ga). The screenbinarization threshold value setting circuit 1701 calculates the screenbinarization threshold value for all the screens and stores thesesequentially in the memory 1702.

The binarization threshold value interpolator 1703 uses the screenbinarization threshold values of all the screens stored in the memory1702 to set binarization threshold values to be applied to each pixel inthe interpolation block BH. In the description given below binarizationthreshold values to be applied to each pixel are set. Note that, theinterpolation block BH is set by the block reading controller 1404,however, depending on the mode of use, it is also possible for theinterpolation block BH to be set by the CPU 108.

The calculation of the binary threshold value to be applied to eachpixel in an interpolation block will now be described in outline. FIG.18A and FIG. 18B show relationships between an interpolation block andscreens. As is clearly shown in the figures, the interpolation block BHbridges four adjacent screens Ga, Gb, Gc, and Gd. Note that the meanvalues of the smoothed photometric values of the screens Ga, Gb, Gc, andGd are set respectively as a, b, c, and d. The binarization thresholdvalue interpolator 1703 uses the smoothed mean values of the photometricvalues a, b, c, and d to calculate the binarization threshold value tobe applied to each pixel within the interpolation block BH.

A description will now be given of the method of calculating thebinarization threshold value to be applied to a pixel bp inside theinterpolation block BH. The shape of the interpolation block BH is takenas being a rectangle with the size (number of pixels) thereof set asxbnum in the horizontal direction and ybnum in the vertical direction.The position of the pixel bp is set as (m, 1). At this time, the valuea×Cb(Ga) is at the point (0, 0) of the interpolation block BH, the valueb×Cb(Gb) is at the point (xbnum, 0) of the interpolation block BH, thevalue c×Cb(Gc) is at the point (0, ybnum) of the interpolation block BH,and the value d×Cb (Gd) is at the point (xbnum, ybnum) of theinterpolation block BH. The area between these points is linearlyapproximated.

In the seventh embodiment, the coefficient was given the fixed value Cb,however, in this ninth embodiment, the coefficient is different for eachscreen. This is necessary in order to make possible higher qualitybinarization than when using the mean value of the photometric values ofeach screen. In particular, when the block division and screen divisionare different, a high quality screen binarization threshold value isset.

The binarization threshold value th to be applied to the pixel bp iscalculated using the aforementioned equation (10). Note that, inequations (9) and (10), a, b, c, and d are replaced with a×Cb(Ga),b×Cb(Gb), c×Cb(Gc), and d×Cb(Gd).

In the binarizer 123, luminance values of the pixels bp in theinterpolation block taken from the frame memory 107 or the block buffer109 are compared with the binarization threshold value for the pixel bpcalculated by the binarization threshold value interpolator 1703.Binarization of the luminance values of the block BH is then performed.The binarized image data then undergoes image compression appropriate toa binarized image, such as MH or MMR, from the compressor 124, in thesame way as in the seventh embodiment.

FIG. 19 is a flowchart showing the flow of image data processing upuntil a multi-valued image is binarized. Firstly, the CPU 108 reads thesize of the image from the CCD 101 (step S1901). Next, the CPU 108 setsone screen G(i, j) from among the photometric values output from thephotometer 1401 (step S1902) and calculates the mean value of thatscreen (step S1903) The mean may be determined using all the photometricvalues or using an appropriate sampling thereof.

Next, a determination is made as to whether or not the mean value of thephotometric values of the screen is inconsistent with the mean values ofthe photometric values of adjacent screens (step S1904). If the meanvalue is inconsistent (i.e. the result of determination in step S1904 isYES), then the mean value of the photometric values is smoothed usingthe smoother 1402 (step S1905). If the mean value is not inconsistent(i.e. the result of determination in step S1904 is NO), oralternatively, if the smoothing processing of step S1905 has beencompleted, the screen binarization threshold value is calculated basedon the mean value of the photometric values (step S1906). This screenbinarization threshold value is then stored in the memory 1702 (stepS1907).

A determination is then made as to whether or not screen binarization ofall blocks has been completed (step S1908). If the screen binarizationof all blocks has not been completed (i.e. if the result ofdetermination in step S1908 is NO), the routine returns to step S1902,the next screen (for example, the screen G(i+1, j)) is set, and stepsS1902 to S1908 are repeated. If screen binarization of all blocks hasbeen completed (i.e. if the result of determination in step S1908 isYES), the interpolation block is set (step S1909). This interpolationblock is suitable for performing partial high quality binarization andmay be set in advance by the user, or the central portion of the imagemay be set via an appropriate mode switching.

The binarization threshold value interpolator 1703 uses the screenbinarization threshold values (e.g. a×Cb(Ga)) of the screens bridged bythe interpolation block set in step S1909 to set the binarizationthreshold values th(x, y) of each single pixel in the interpolationblock (step S1910).

A pixel (here referred to as g(x, y)) read from the frame memory 107 bythe interpolator 123 is binarized using the binarization threshold valuecalculated in step S1910 (step S1911). Next, a determination is made asto whether or not binarization processing has been performed on all thepixels (step S1912). If binarization has not been performed on all thepixels (i.e. if the result of determination in step S1912 is NO), apixel in the interpolation block adjacent to g(x, y) (for example,g(x+1, y)) is set, and steps S1910 to S1912 are repeated. Ifbinarization has been performed on all the pixels (i.e. if the result ofdetermination in step S1912 is YES), a determination is made as towhether or not binarization has been completed for all interpolationblocks (step S1913). If binarization has not been completed for allinterpolation blocks (i.e. if the result of determination in step S1913is NO), then steps S1909 to S1913 are repeated. If binarization has beencompleted for all interpolation blocks (i.e. if the result ofdetermination in step S1913 is YES), then processing is ended.

In this ninth embodiment, because a binarization threshold value is setusing information (photometric values) obtained from an automaticphotometric section provided in the digital camera, separate processingin order to set a binarization threshold value is unnecessary. As aresult, the circuit structure can be simplified and costs can bereduced. At the same time, because a separate binarization thresholdvalue counting processing is unnecessary, power consumption can bereduced. In addition, even if the image has pinpoint reflections from alight source, high quality binarization of a multi-valued image ispossible. Furthermore, because binarization threshold values to beapplied to each pixel in an interpolation block are set and binarizationis performed on the basis of photometric values, higher quality imageprocessing than from the digital camera of the seventh embodiment ispossible. In particular, when photographing a notice board or the like,when all of the characters are small or when characters in a portion ofthe area are small, it is possible to perform partial high qualitybinarization in those locations.

The tenth embodiment explained below relaters to an image pickupapparatus which calculates binarization threshold values to be appliedto each pixel in a predetermined block and performs binarization of theimage data for each pixel using a CMOS sensor and a photometer is usedin a digital camera. FIG. 20 is a structural diagram showing an exampleof a case in which the image pickup apparatus which calculatesbinarization threshold values to be applied to each pixel in apredetermined block and performs binarization of the image data for eachpixel using a CMOS sensor and a photometer is used in a digital camera.Note that those portions of the present embodiment which are identicalto those of the ninth embodiment are given the same descriptive symbolsand a description thereof is omitted. The description given here ismainly of those portions that differ from the ninth embodiment.

The digital camera 2000 is provided with a CMOS sensor 2001 in the imageinput section thereof. As a result, unlike the CCD 101 (see FIG. 17)which can only perform raster scanning, because the CMOS sensor 2001 iscapable of random access and reading of single block units, the framememory 107 and block reading controller 1404 are unnecessary, allowingthe circuit structure to be simplified. In this case, the CPU 108performs the functions of the block reading controller.

Further, in contrast, to the CCD 101 which requires a separate powersource to the CMOS integrated circuit, the CMOS sensor 2001 is able touse the same power source as the CMOS integrated circuit, enabling powerconsumption to be reduced. As a result, because the scale of thecircuitry of the digital camera 2000 can also be reduced, this system isfar more efficient in terms of power consumption, processing speed andcost than a system using a CCD. Note that, in the present embodiment, aCMOS sensor was employed, however, any other image pickup apparatushaving a block accessible image input section may be used.

The eleventh embodiment explained below relates to an image pickupapparatus, in which a binarization threshold value is set for eachobject block based on photometric values output from photometry unitafter those photometric values have been adjusted so as to be within apredetermined range, is used in a digital camera. FIG. 21 is a blockdiagram showing an example of the apparatus structure from image inputuntil a binarized image is recorded when an image pickup apparatus, inwhich a binarization threshold value is set for each object block basedon photometric values output from photometry unit after thosephotometric values have been adjusted so as to be within a predeterminedrange, is used in a digital camera.

The digital camera 2100 is provided with the CCD 101, the A/D converter102, the white balance adjustor 103, the pixel interpolator 104, theluminance generator 105, the aperture corrector 106, the frame memory107, the CPU 108, a photometer 2101, a smoother 2102, a limiter 2105,memory 2103, a block reading controller 2104, the binarization thresholdvalue setting circuit 122, the binarizer 123, the compressor 124, andthe image storage memory 125. Note that a section for generating colordifference signals is omitted.

The photometer 2101 is provided with an automatic exposure (AE)detecting mechanism for performing photometry on an object to bephotographed before photography of the image. The photometer 2101measures the brightness of each screen based on digital signals outputfrom the A/D converter 102. The method of the photometry may, forexample, comprise measurement by adding the luminance values of thepixels. At this time, the CPU 108 can also sample photometric values tobe used in the addition by the photometer 2101.

The smoother 2102 smoothes the photometric values of each screenobtained by the photometer 2101 and outputs them as ave(i, j) to thelimiter 2105. The processing below is an example of this smoothing.Namely, when the mean value ave(i, j) of the photometric values of allthe pixels contained in one screen (this screen is referred to here asG(i, j)) within the photometer 2101 (alternatively, the mean value ofthe photometric values sampled from within the screen G(i, j)) isinconsistent with the mean values of the photometric values of thesurrounding screens, the photometric values of each pixel of the screenG(i, j) are corrected so that they are consistent with the mean valuesof the photometric values of the surrounding screens.

An example of an algorithm for performing this processing is givenbelow. The mean photometric value of the four screens adjacent to thescreen G(i, j) is set as ave4(i, j). If the mean photometric valueave(i, j) of the screen G(i, j) is three times the value ave4(i, j) ormore, then the mean photometric values ave(i, j) of the screen G(i, j)are converted using the following equation (12): $\begin{matrix}{{{{if}\quad{{ave}\left( {i,j} \right)}} \geq {3 \times {{ave4}\left( {i,j} \right)}}}{{{then}\quad{{ave}\left( {i,j} \right)}} = {{{ave4}\left( {i,j} \right)} + {\left( {1/4} \right) \times \left( {{{ave}\left( {i,j} \right)} - {{ave4}\left( {i,j} \right)}} \right)}}}} & (12)\end{matrix}$

In the smoother 2102, the mean photometric values ave(i, j) of thescreen G(i, j) are calculated as described above. Furthermore, bylimiting the ave(i, j) to a predetermined lower limit value in thelimiter 2105, mean photometric values that are too low are preventedfrom being output. Then, the mean photometric value is output to thememory 2103. Note that, in the present embodiment, depending on the modeof use, it may also be possible to output photometric values whose rangeis limited by the limiter 2105 without providing a smoother 2102.

The binarization threshold value setting circuit 122 sets thebinarization threshold TH(i, j) based on the ave(i, j) The setting unitis a multiplier for multiplying the ave(i, j) by a preset coefficientCb, however, if the coefficient Cb is made to equal x/16 or Cb is madeto equal x/8 (wherein x is a predetermined value representing a naturalnumber no greater than the denominator), then the binarization thresholdvalue setting circuit 122 can be constructed simply from an adder, whichis efficient in terms of both cost and speed.

The binarizer 123 binarizes each pixel of the block B(i, j) whichcorresponds to the screen G(i, j) based on the binarization thresholdvalue TH(i, j). The compressor 124 performs compression appropriate to abinary image such as MH or MR. The compressed image is then stored inthe image storage memory 125.

Next, the flow of the processing as far as the binarization of amulti-valued image will be described. FIG. 22 is a flow chart showingthe flow of the image data processing as far as the binarization of amulti-valued image. Firstly, the CPU 108 reads the size of the imagefrom the CCD 101 (step S2201). Next, the CPU 108 sets one screen G(i, j)from among the photometric values output from the photometer 2101 (stepS2202) and calculates the mean value of that screen (step S2203) All thephotometric values or a suitable sampling thereof may be used todetermine the mean.

Next, a determination is made as to whether or not the mean value of thephotometric values of the screen is inconsistent with the mean values ofthe photometric values of adjacent screens (step S2204). If the meanvalue is inconsistent (i.e. the result of determination in step S2204 isYES), then the mean value of the photometric values is smoothed usingthe smoother 2102 (step S2205). If the mean value is not inconsistent(i.e. the result of determination in step S2204 is NO), oralternatively, if the smoothing processing of step S2205 has beencompleted, then a determination is made as to whether or not the outputphotometric values are too low (step S2206). If the output photometricvalues are too low (i.e. the result of determination in step S2206 isYES), the photometric values are replaced with preset values (stepS2207). If the output photometric values are not too low (i.e. if theresult of determination in step S2206 is NO), or alternatively, if thevalues have been replaced in step S2207, then a binarization thresholdvalue is calculated based on the photometric values of the screen G(i,j) (step S2208).

Next, the block B(i, j) which corresponds to the screen G(i, j) is readfrom the frame memory 107 by the block reading controller 2104 (stepS2209), and the multi-valued image within the block is binarized usingthe binarization threshold value (step S2210). A determination is thenmade as to whether or not binarization of all the blocks has beencompleted (step S2211). If binarization of all the blocks has beencompleted (i.e. if the result of determination in step S2211 is YES),the binarization processing is ended. If binarization of all the blockshas not been completed (i.e. if the result of determination in stepS2211 is NO), the routine returns to step S2202, the next screen is set(for example, G(i+1, j)), and the steps 2202 to S2211 are repeated.

In this eleventh embodiment, because a binarization threshold value isset using information (photometric values) obtained from an automaticphotometry section provided in a digital camera, there is no need forseparate processing to set the binarization threshold value. As aresult, the circuit structure is simplified and the cost thereofreduced. Further, because separate binarization threshold valuecalculation processing is unnecessary, power consumption can be reduced.In addition, even if the image has pinpoint reflections from a lightsource, high quality binarization of a multi-valued image is possible.Moreover, by using a smoother and a limiter in combination, binarizationthreshold values are set efficiently.

Note that a CMOS sensor 101 is used in the digital camera 2100, however,depending on the mode of use, a structure in which a CMOS sensor 2301 isused in the digital camera 2300, as is shown in FIG. 23, may beemployed. By employing this type of structure, frame memory and a block,reading controller become unnecessary and the circuit structure issimpler in comparison to the digital camera 2100. Moreover, the CMOSsensor 2301 can use the same power source as a CMOS integrated circuit,enabling power consumption to be reduced. As a result, the scale of thecircuitry of the digital camera 2300 can be made smaller than that ofthe digital camera 2100 and the power consumption is also smaller.Consequently, it is possible to provide a more efficient digital camerathan the digital camera 2100 in terms of power consumption, processingspeed and cost.

The twelfth embodiment explained below relates to an image pickupapparatus which calculates binarization threshold values to be appliedto each individual pixel within a predetermined block based onphotometric values which have been limited to a predetermined range, isused in a digital camera. FIG. 24 is a diagram showing an example of thestructure when such an image pickup apparatus is used in a digitalcamera.

The digital camera 2400 is provided with the CCD 101, the A/D converter102, the white balance adjustor 103, the pixel interpolator 104, theluminance generator 105, the aperture corrector 106, the frame memory107, the CPU 108, the photometer 2101, the smoother 2102, a limiter2401, the screen binarization threshold value setting circuit 1701, thememory 1702, the binarization threshold value interpolator 1703, thebinarizer 123, the compressor 124, and the image storage memory 125.Note that a section for generating color difference signals is omitted.

The CCD 101 converts light converged by a not shown optical system ofthe digital camera 2400 into electric signals and outputs R, G, B analogsignals for each pixel forming a multi-valued image. The output analogsignals are converted into digital signals by the A/D converter 102. Asthey pass through the white balance adjustor 103, the pixel interpolator104, the luminance generator 105, and the aperture corrector 106, thedigital signals undergo processing such as luminance valueinterpolation, extraction, and the like and are then temporarily storedin the frame memory 107.

The CPU 108 controls each of the circuits and members of the digitalcamera 2400. The photometer 2101 is provided with the automatic exposure(AE) detection mechanism for performing photometry on an object beforean image is photographed, and measures the brightness of each screenbased on digital signals output from the A/D converter 102. The methodof the photometry may, for example, comprise measurement by adding theluminance values of the pixels. At this time, the CPU 108 can alsosample photometric values to be used in the addition by the photometer2101.

The smoother 2102 smoothes the photometric values of each screenobtained by the photometer 2101 and outputs them as ave(i, j) to thelimiter 2105. In the limiter 2401, the ave(i, j) is limited by a presetlower limit value so that mean photometric values which are too low areprevented from being output.

The screen binarization threshold setting circuit 1701 sets thebinarization threshold value TH(i, j) on the basis of the ave(i, j). Theblock binarization threshold values for all the blocks are stored in thememory 1702. The binarization threshold values for individual pixels arenext calculated by the binarization threshold value interpolator 1703using the block binarization threshold values for all the blocks. Theluminance values of each pixel are read simultaneously from the framememory 107 while the binarization threshold values for the pixels arebeing calculated. The luminance values and the binarization thresholdvalues are input into the binarizer 123. In the binarizer 123, thebinarization threshold values and the luminance values are compared andthe luminance values are binarized. The binarized image then undergoesimage compression in the compressor 124 appropriate to a binarized imagesuch as MH or MMR. The compressed image is then stored in the imagestorage memory 125. Note that a digital camera is achievable with thesame type of structure even when a CMOS sensor is used instead of theCCD 101.

Next, the flow of the processing until a multi-valued image is binarizedwill be described. FIG. 25 is a flow chart showing the flow of imagedata processing until a multi-valued image is binarized. Firstly, theCPU 108 reads the size of the image from the CCD 101 (step S2501). Next,the CPU 108 sets one screen G(i, j) from among the photometric valuesoutput from the photometer 2101 (step S2502) and calculates the meanvalue of that screen (step S2503). The mean may be determined using allthe photometric values or using an appropriate sampling thereof.

Next, a determination is made as to whether or not the mean value of thephotometric values of the screen is inconsistent with the mean values ofthe photometric values of adjacent screens (step S2504). If the meanvalue is inconsistent (i.e. the result of determination in step S2504 isYES), then the mean value of the photometric values is smoothed usingthe smoother 2102 (step S2505). If the mean value is not inconsistent(i.e. the result of determination in step S2504 is NO), oralternatively, if the smoothing processing of step S2505 has beencompleted, a determination is then made as to whether or not thephotometric values are too low (step S2506). If the photometric valuesare too low (i.e. if the result of determination instep S2506 is YES),those photometric values are replaced with predetermined values (stepS2507). If the photometric values are not too low (i.e. if the result ofdetermination in step S2506 is NO), or alternatively, if the replacementprocessing of step S2507 has been completed, the block binarizationthreshold value TH(i, j) of the block corresponding to the screen iscalculated based on the photometric values of the screen G(i, j) (stepS2508).

A determination is then made as to whether or not a block binarizationthreshold value has been calculated for all the blocks (step S2509). Ifthe block binarization threshold values of all the blocks have beencalculated (i.e. if the result of determination in step S2509 is YES),the processing routing moves to the next step. If the block binarizationthreshold values of all the blocks has not been calculated (i.e. if theresult of determination in step S2509 is NO), then the block adjacent tothe block G(i, j) (for example, G(i+1, j)) is set and steps S2502 toS2509 are repeated.

When block binarization threshold values have been calculated for allthe blocks (i.e. when the result of determination in step S2509 is YES),an interpolation block is set (step S2510). The binarization thresholdvalue interpolator 1703 then calculates binarization threshold valuesth(x, y) of each individual pixel by interpolation using thebinarization threshold values (step S2511) Note that x and y are naturalnumbers which represent the position of each pixel in an image.Furthermore, the pixels read from the frame memory 107 (here taken asg(x, y)) are binarized using the above binarization threshold valuesth(x, y) (step S2512). A determination is then made as to whether or notbinarization processing has been completed for all the pixels (stepS2513) If binarization processing has not been completed for all thepixels (i.e. if the result of determination in step S2513 is NO), then apixel adjacent to the pixel g(x, y) (e.g. the pixel g(x+1, y)) is setand steps S2511 to S2513 are repeated.

If, however, binarization processing has been completed for all thepixels (i.e. if the result of determination in step S2513 is YES), adetermination is then made as to whether or not binarization has beencompleted in all the interpolation blocks (step S2514). If binarizationhas not been completed in all the interpolation blocks (i.e. if theresult of determination in step S2514 is NO), then steps S2510 to S2514are repeated. If binarization has been completed in all theinterpolation blocks (i.e. if the result of determination in step S2514is YES), the processing is ended.

In this twelfth embodiment, because binarization threshold values areset using information (photometric values) obtained from an automaticphotometric section provided in a digital camera, separate processingfor setting binarization threshold values is unnecessary. As a result,the circuit structure can be simplified and costs reduced. Further,because separate binarization threshold value calculation processing isunnecessary, power consumption can also be reduced. In addition, even ifpinpoint reflections due to a light source are present in an image, highquality binarization of a multi-valued image is possible. Furthermore,by using a smoother and limiter in combination, binarization thresholdvalues are efficiently set. Moreover, because binarization thresholdvalues to be applied to each individual pixel in an interpolation blockare set and binarization performed based on photometric values, highquality image processing is possible. In particular, when photographingnotice boards, in cases such as when each character is small orcharacters in a portion of the area are small, high quality partialbinarization of the relevant areas is possible.

The thirteenth embodiment explained below relates to an image pickupapparatus which performs image binarization by software processing inthe CPU. FIG. 26 is a block diagram showing an example of the apparatusstructure from input of an image until a binarized image is recordedwhen an image pickup apparatus which performs image binarization bysoftware processing in the CPU is used in a digital camera. Note that,in the present embodiment, structural elements that are identical tothose described in each of the previously described embodiments aregiven the same descriptive symbols and a detailed description thereof isomitted. The description given here is mainly of those portions whichare different to previous embodiments.

The digital camera 2600 comprises the CCD 101, the A/D converter 102,the white balance adjustor 103, the pixel interpolator 104, theluminance generator 105, the aperture corrector 106, the frame memory107, a CPU 2601, ROM 2602, RAM 2603, and the image storage memory 125. Asection for generating color difference signals is omitted.

The CCD 101 converts light converged by a not shown optical system ofthe digital camera 2600 into electrical signals and outputs R, G, Banalog signals of each pixel constituting the multi-valued image. Theoutputted analog signals are converted into digital signals by the A/Dconverter 102. The digital signals then undergo various processing suchas interpolation of the luminance values, extraction and the like asthey pass through the white balance adjustor 103, the pixel interpolator104, the luminance generator 105, and the aperture corrector 106, andare then stored temporarily in the frame memory 107.

The CPU 2601 controls each of the circuits and members of the digitalcamera 2600 and has the function of binarizing the image. A softwareprogram for performing the binarization function is stored in the ROM2602 connected to the CPU2601. The RAM 2603 stores data such as imagedata and the like and acts as a work area for performing thebinarization processing.

The software program for performing the functions of the mean luminancecalculator, the low luminance threshold value setter, the binarizationthreshold value setting circuit, the binarization threshold valueinterpolator, the binarizer, the compressor, and the like which havebeen described in the previous embodiments is stored in the ROM 2602.The mean luminance calculation, the binarization threshold valuesetting, the binarization, the compression of the binarized image andthe like are performed in the CPU 2601 through the program being run. Animage that has been binarized and compressed in the CPU 2601 is storedin the image storage memory 125.

Depending on the mode of use, a software program for performing thefunctions of the photometer, the smoother, the limiter, the blockbinarization threshold value setting circuit, the binarization thresholdvalue interpolator, the binarizer, the compressor, and the like may bestored in the ROM 2602 in the digital camera 2600, and the binarizationthreshold value setting, the binarization, the compression of thebinarized image and the like can be performed by the running of theprogram in the CPU 2601. An image that has been binarized and compressedin the CPU 2601 is stored in the image storage memory 125. It is alsopossible to use a CMOS sensor instead of the CCD 101.

In this thirteenth embodiment, because each function is performed by theCPU 2601 and the ROM 2602, there is no need for a separate section toperform each of the functions and development costs can be reduced. Thisallows a low cost digital camera to be provided. Moreover, by upgradingthe software, it is possible to constantly provide new algorithms.

The fourteenth embodiment explained below relates to an image pickupapparatus provided with a photometer for performing image binarizationin the CPU by software processing. FIG. 27 is a block diagram showingthe apparatus structure from input of an image until a binarized imageis recorded when an image pickup apparatus which performs imagebinarization by software processing in the CPU is used in a digitalcamera. Note that, in the present embodiment, a detailed description ofstructural elements that are identical to those of the thirteenthembodiment is omitted. The description given here is mainly of thoseportions that are different to the thirteenth embodiment.

The digital camera 2700 comprises the CCD 101, the A/D converter 102,the white balance adjustor 103, the pixel interpolator 104, theluminance generator 105, the aperture corrector 106, the frame memory107, a CPU 2701, ROM 2702, RAM 2703, a photometer 2704, memory 2705, andthe image storage memory 125. A section for generating color differencesignals is omitted.

The CCD 101 converts light converged by a not shown optical system ofthe digital camera 2700 into electrical signals and outputs R, G, Banalog signals of each pixel constituting the multi-valued image. Theoutputted analog signals are converted into digital signals by the A/Dconverter 102. The digital signals then undergo various processing suchas interpolation of the luminance values, extraction, and the like asthey pass through the white balance adjustor 103, the pixel interpolator104, the luminance generator 105, and the aperture corrector 106, andare then stored temporarily in the frame memory 107.

The photometer 2704 is provided with automatic exposure (AE) detectionmechanism for performing photometry on an object before an image isphotographed, and measures the brightness of each screen based ondigital signals output from the A/D converter 102. The method of thephotometry is measurement by adding the luminance values of the pixels.At this time, it is also possible for the CPU 2701 to sample thephotometric values to be used in the addition by the photometer 2704.Because it is possible to generate luminance signals at the same time ascalculate mean luminance values in the photometer 2704, the amount ofthe calculation in the CPU 2701 is reduced, and the processing from thephotography of the image to the storage of the binary image can be morerapidly performed compared to the thirteenth embodiment.

In addition to the function of controlling the each of the circuits andmembers of the digital camera 2700, the CPU 2701 is provided with animage binarization function. A software program for performing thebinarization function is stored in the ROM 2702 connected to the CPU2701. The RAM 2703 stores data such as image data and the like and actsas a work area for performing the binarization processing.

A software program for performing the functions of the smoother, thelimiter, the block binarization threshold value setting circuit, thebinarization threshold value interpolator, the binarizer, thecompressor, and the like may be stored in the ROM 2702, and thebinarization threshold value setting, the binarization, the compressionof the binarized image and the like can be performed by the running ofthe program in the CPU 2701. An image that has been binarized andcompressed in the CPU 2701 is stored in the image storage memory 125.Depending on the mode of use, it is also possible to use a CMOS sensorinstead of the CCD 101.

In this fourteenth embodiment, because each function is performed by theCPU 2701 and the ROM 2702, in the same way as in the thirteenthembodiment, there is no need for a separate section to perform each ofthe functions and development costs can be reduced. This allows a lowcost digital camera to be provided. Moreover, by upgrading the software,it is possible to constantly provide new algorithms.

The present invention can also be achieved using software in addition tothe above-described embodiments. FIG. 28 shows an example of thecomputer system which can realized the present invention using software.

In FIG. 28, CPU 2801 performs the overall control of the apparatus basedon a control program; ROM 2802 stores the control program; 2803 is RAM;display device 2804 displays input and output states of a computer; 2805is a hard disk; keyboard 2806 is used to input character strings and thelike; 2807 is a CD-ROM drive. CD-ROM 2808 serves as a computer readablestorage medium on which a program for executing the image binarizationmethod of the present invention is recorded.

In the computer system having the above structure, a program forexecuting the image binarization method of the present invention isstored on the CD-ROM 2808. This program is read and activated by thecontrol and processing of the CPU 2801, thus allowing image binarizationprocessing to be performed. The binarized information is output to thehard disk 2805 or the like.

As has been described above, with the image binarization apparatusaccording to one aspect of the present invention, low luminance valuescan be removed from an object block based on mean luminance values ofthe surrounding blocks, and binarization threshold values can be set forthe object blocks based on the mean of the luminance values from whichthe low luminance values have been removed. Consequently, high qualitybinarization of a multi-valued image is possible.

With the image binarization apparatus according to another aspect of thepresent invention, low luminance values can be removed based on meanluminance values of the surrounding blocks, and binarization thresholdvalues can be set for the object blocks based on the mean of theluminance values from which the low luminance values have been removed.Moreover, binarization threshold values to be applied to each pixelwithin an interpolation block can be set based on binarization thresholdvalues of adjacent object blocks. Consequently, high qualitybinarization of a multi-valued image is possible.

With the image binarization apparatus according to still another aspectof the present invention, low luminance values can be removed from anobject block based on mean luminance values of the surrounding blocks,and the mean of the luminance values from which the low luminance valueshave been removed can be rounded to values within a predetermined range.Moreover, binarization threshold values can be set for the object blocksbased on these values. Consequently, high quality binarization of amulti-valued image is possible.

With the image binarization apparatus according to still another aspectof the present invention, low luminance values can be removed based onmean luminance values of the surrounding blocks, and the mean of theluminance values from which the low luminance values have been removedcan be rounded to values within a predetermined range. Moreover,binarization threshold values can be set for the object blocks based onthese values and binarization threshold values to be applied to eachpixel within an interpolation block can be set based on binarizationthreshold values of adjacent object blocks. Consequently, high qualitybinarization of a multi-valued image is possible.

Further, the size of created blocks is changed in accordance with theimage size or with the total number of pixels of the multi-valued image.As a result, a suitable block size for the size of the characters can beselected in accordance with the image size or with the total number ofpixels of the multi-valued image. Consequently, high qualitybinarization of a multi-valued image is possible.

Further, the size or shape of blocks to be created is changed inaccordance with the positions of blocks to be created within themulti-valued image. As a result, detailed corrections which arising fromthe optical system such as peripheral light reduction can be performed.Consequently, high quality binarization of a multi-valued image ispossible.

Further, the image binarization apparatus comprises a sampling unitwhich samples pixels which form the multi-valued image, and theluminance value output unit outputs luminance values of pixels sampledby the sampling unit so that the number of pixels when a mean luminancevalue in a block is calculated can be reduced. Consequently, highquality binarization of a multi-valued image can be performed rapidlyand at a low level of power consumption.

Further, the sampling unit sets a sampling interval used in the samplingin accordance with the image size, the total number of pixels, or theblock size. As a result, even if the block size changes, the number ofpixels can be reduced or kept constant when calculating mean luminancevalues within object blocks. Consequently, high quality binarization ofa multi-valued image can be performed rapidly and at a low level ofpower consumption.

Further, the mean luminance value calculation unit comprises an addingunit for adding the luminance values of each pixel and a counting unitfor counting the number of pixels added by the adding unit, and when thenumber of pixels counted by the counting unit is a power of two, theadding unit determines a mean luminance value. As a result, a divider isnot necessary when calculating a mean value and a simple structure usingonly an adder can be employed. Consequently, high quality binarizationof a multi-valued image can be performed rapidly and at a low level ofpower consumption.

With the image pickup apparatus according to still another aspect of thepresent invention, binarization threshold values of created blocks canbe set on the basis of smoothed photometric values of the createdscreens. Consequently, high quality binarization of a multi-valued imageis possible.

With the image pickup apparatus according to still another aspect of thepresent invention, binarization threshold values to be applied to eachpixel of an interpolation block can be set on the basis of smoothedphotometric values of adjacent created screens. Consequently, highquality binarization of a multi-valued image is possible.

With the image pickup apparatus according to still another aspect of thepresent invention, the photometric values of the created screens can berounded to within a predetermined range and the binarization thresholdvalues of the created blocks can be set based on these values.Consequently, high quality binarization of a multi-valued image ispossible.

With the image pickup apparatus according to still another aspect of thepresent invention, the photometric values of the adjacent createdscreens can be rounded to within a predetermined range, and thebinarization threshold values to be applied to each pixel of theinterpolation blocks can be set based on these values. Consequently,high quality binarization of a multi-valued image is possible.

Further, the screens created by the screen division unit are identicalto blocks created by the block division unit. As a result, it ispossible to enhance the correlation between a photometric value of ascreen and a luminance value of a block. Consequently, high qualitybinarization of a multi-valued image is possible.

With the image binarization method according to still another aspect ofthe present invention low luminance values of object blocks can beremoved on the basis of mean luminance values of surrounding blocks, andbinarization threshold values of the object blocks can be set on thebasis of the mean of the luminance values from which the low luminancevalues have been removed. Consequently, high quality binarization of amulti-valued image is possible.

With the image binarization method according to still another aspect ofthe present invention, low luminance values can be removed on the basisof mean luminance values of surrounding blocks, and binarizationthreshold values of the object blocks can be set on the basis of themean of the luminance values from which the low luminance values havebeen removed. In addition, binarization threshold values to be appliedto each pixel within an interpolation block can be set based onbinarization threshold values of adjacent object blocks. Consequently,high quality binarization of a multi-valued image is possible.

With the image binarization method according to still another aspect ofthe present invention, low luminance values of object blocks can beremoved on the basis of mean luminance values of surrounding blocks, andthe unit of the luminance values from which low luminance values havebeen removed can be rounded to within a predetermined range. Inaddition, binarization threshold values of object blocks can be setbased on these values. Consequently, high quality binarization of amulti-valued image is possible.

With the image binarization method according to still another aspect ofthe present invention, low luminance values can be removed on the basisof mean luminance values of surrounding blocks, and the unit of theluminance values from which low luminance values have been removed canbe rounded to within a predetermined range. In addition, binarizationthreshold values of object blocks can be set based on these values andbinarization threshold values to be applied to each pixel of aninterpolation block can be set based on binarization threshold values ofadjacent object blocks. Consequently, high quality binarization of amulti-valued image is possible.

Further, in the block division step, the size of an object block ischanged in accordance with the image size of the multi-valued image orwith the total number of pixels of the multi-valued image. As a result,it is possible to select a block size appropriate to the size of thecharacters in accordance with the image size or total number of pixels.Consequently, high quality binarization of a multi-valued image ispossible.

Further, in the block division step, the size or shape of blocks to becreated is changed in accordance with the positions of blocks to becreated within the multi-valued image. As a result, detailed correctionsarising from the optical system such as peripheral light reduction arepossible. Consequently, high quality binarization of a multi-valuedimage is possible.

Further, a sampling step for sampling pixels which form the multi-valuedimage is provided and, in the low luminance value removal step, usingpixels sampled in the sampling step, luminance values below the lowluminance threshold value are removed from luminance values of thepixels and only luminance values which exceed the threshold luminancevalue are output. As a result, the number of pixels used whencalculating a mean luminance value within a block can be reduced.Consequently, high quality binarization of a multi-valued image can beachieved rapidly and at a low level of power consumption.

Further, a sampling step for sampling pixels which form the multi-valuedimage is provided and, in the mean luminance value calculation step,mean luminance values are calculated using pixels sampled in thesampling step. As a result, the number of pixels used when calculating amean luminance value within a block can be reduced. Consequently, highquality binarization of a multi-valued image can be achieved rapidly andat a low level of power consumption.

Further, in the sampling step, a sampling interval used in the samplingis set in accordance with the image size, the total number of pixels, orthe block size. As a result, the number of pixels used when calculatinga mean luminance value within a block can be reduced or kept constanteven if the size of the block changes. Consequently, high qualitybinarization of a multi-valued image can be achieved rapidly and at alow level of power consumption.

With the image pickup method according to still another aspect of thepresent invention, binarization threshold values of created blocks canbe set on the basis of smoothed photometric values of a created screen.Consequently, high quality binarization of a multi-valued image ispossible.

With the image pickup method according to still another aspect of thepresent invention, binarization threshold values applied to each pixelof an interpolation block are set on the basis of smoothed photometricvalues of adjacent created screens. Consequently, high qualitybinarization of a multi-valued image is possible.

With the image pickup method according to still another aspect of thepresent invention, photometric values of created screens are rounded tovalues within a predetermined range and binarization threshold values ofcreated blocks are set based on these values. Consequently, high qualitybinarization of a multi-valued image is possible.

With the image pickup method according to still another aspect of thepresent invention, photometric values of adjacent created screens arerounded to values within a predetermined range and binarizationthreshold values applied to each pixel of the interpolation blocks areset based on these values. Consequently, high quality binarization of amulti-valued image is possible.

Further, the screens created in the screen division step are identicalto blocks created in the block division step. As a result, thecorrelation between the screen photometric values and the blockluminance values is increased. Consequently, high quality binarizationof a multi-valued image is possible.

The computer readable recording medium according to still another aspectof the present invention enables a computer to function on the basis ofeach step of the image binarization method described above.

The present document incorporated by reference the entire contents ofJapanese priority document, 11-113761 filed in Japan on Apr. 21, 1999and 2000-035946 filed in Japan on Feb. 14, 2000.

Although the invention has been described with respect to a specificembodiment for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art which fairly fall within the basic teaching hereinset forth.

1-68. (canceled)
 69. An image binarization apparatus comprising: a blockdivision unit which divides a multi-valued image into blocks; aluminance value output unit which outputs a luminance value of eachpixel forming the multi-valued image; a binarization threshold valuesetting unit which sets a binarization threshold value to be used whenbinarizing the multi-valued image; a low luminance threshold valuesetting unit which sets a low luminance threshold value to be used whenremoving low luminance values; a low luminance value removal unit whichinputs luminance values of each pixel from among luminance values outputby said luminance value output unit, removes luminance values that arelower than the low luminance threshold value set by said low luminancethreshold value setting unit, and outputs only those luminance valueswhich exceed the low luminance threshold value; and a mean luminancevalue calculation unit which calculates a mean luminance value of theluminance values output by said low luminance value removal unit,wherein said low luminance threshold value setting unit sets the lowluminance threshold value based on mean luminance values of at leastselected of the blocks; and said binarization threshold value settingunit sets the binarization threshold values based on mean luminancevalues of the blocks.
 70. The image binarization apparatus according toclaim 69, wherein said block division unit changes the size of createdblocks in accordance with the image size of the multi-valued image orwith the total number of pixels of the multi-valued image.
 71. The imagebinarization apparatus according to claim 69, wherein said blockdivision unit changes the size or shape of blocks to be created inaccordance with the positions of blocks to be created within themulti-valued image.
 72. The image binarization apparatus according toclaim 69 further comprising a sampling unit which samples pixels whichform the multi-valued image, wherein said luminance value output unitoutputs luminance values of pixels sampled by said sampling unit. 73.The image binarization apparatus according to claim 72, wherein saidsampling unit sets a sampling interval used in the sampling inaccordance with the image size, the total number of pixels, or the blocksize.
 74. The image binarization apparatus according to claim 69,wherein said mean luminance value calculation unit comprises an addingunit which adds the luminance values of each pixel; and a counting unitwhich counts the number of pixels added by the adding unit, and when thenumber of pixels counted by said counting unit is a power of two, saidadding unit determines a mean luminance value.
 75. An image binarizationapparatus comprising: a block division unit which divides a multi-valuedimage into blocks; a luminance value output unit which outputs aluminance value of each pixel forming the multi-valued image; abinarization threshold value setting unit which sets a binarizationthreshold value to be used when binarizing the multi-valued image; a lowluminance threshold value setting unit which sets a low luminancethreshold value to be used when removing low luminance values; a lowluminance value removal unit which inputs luminance values of each pixelfrom among luminance values output by said luminance value output unit,removes luminance values that are lower than the low luminance thresholdvalue set by said low luminance threshold value setting unit, andoutputs only those luminance values which exceed the low luminancethreshold value; a mean luminance value calculation unit whichcalculates a mean luminance value of the luminance values output by saidlow luminance value removal unit; a block binarization threshold valuesetting unit which sets a block binarization threshold value which is abinarization threshold value applied to at least selected of the blocksbased on the mean luminance values calculated by said mean luminancevalue calculation unit; and an interpolation block setting unit forsetting interpolation blocks which cover pixels extending over two ormore adjacent of the selected of the blocks, wherein said low luminancethreshold value setting unit sets the low luminance threshold valuebased on mean luminance values of the two or more adjacent of theselected of the blocks; and said binarization threshold value settingunit sets the binarization threshold value to be applied to pixelsinside the interpolation block based on block binarization thresholdvalues of each of the two or more adjacent of the selected of the blocksbridged by the interpolation block.
 76. The image binarization apparatusaccording to claim 75, wherein said block division unit changes the sizeof created blocks in accordance with the image size of the multi-valuedimage or with the total number of pixels of the multi-valued image. 77.The image binarization apparatus according to claim 75, wherein saidblock division unit changes the size or shape of blocks to be created inaccordance with the positions of blocks to be created within themulti-valued image.
 78. The image binarization apparatus according toclaim 75 further comprising a sampling unit which samples pixels whichform the multi-valued image, wherein said luminance value output unitoutputs luminance values of pixels sampled by said sampling unit. 79.The image binarization apparatus according to claim 78, wherein saidsampling unit sets a sampling interval used in the sampling inaccordance with the image size, the total number of pixels, or the blocksize.
 80. The image binarization apparatus according to claim 75,wherein said mean luminance value calculation unit comprises an addingunit which adds the luminance values of each pixel; and a counting unitwhich counts the number of pixels added by the adding unit, and when thenumber of pixels counted by said counting unit is a power of two, saidadding unit determines a mean luminance value.
 81. An image binarizationapparatus comprising: a block division unit which divides a multi-valuedimage into blocks; a luminance value output unit which outputs aluminance value of each pixel forming the multi-valued image; abinarization threshold value setting unit which sets a binarizationthreshold value to be used when binarizing the multi-valued image; amean luminance value calculation unit which receives luminance values ofeach pixel forming at least selected of the blocks from among luminancevalues output by said luminance value output unit, and calculates meanluminance values of the selected of the blocks; and a luminance valuelimiting unit which limits a range of mean luminance values calculatedby said mean luminance value calculation unit so that the values arewithin a predetermined spread, wherein said binarization threshold valuesetting unit sets binarization threshold values of the selected of theblocks based on the mean luminance values the range of which is limitedby said luminance value limiting unit.
 82. The image binarizationapparatus according to claim 81, wherein said block division unitchanges the size of created blocks in accordance with the image size ofthe multi-valued image or with the total number of pixels of themulti-valued image.
 83. The image binarization apparatus according toclaim 81, wherein said block division unit changes the size or shape ofblocks to be created in accordance with the positions of blocks to becreated within the multi-valued image.
 84. The image binarizationapparatus according to claim 81 further comprising a sampling unit whichsamples pixels which form the multi-valued image, wherein said luminancevalue output unit outputs luminance values of pixels sampled by saidsampling unit.
 85. The image binarization apparatus according to claim84, wherein said sampling unit sets a sampling interval used in thesampling in accordance with the image size, the total number of pixels,or the block size.
 86. The image binarization apparatus according toclaim 81, wherein said mean luminance value calculation unit comprisesan adding unit which adds the luminance values of each pixel; and acounting unit which counts the number of pixels added by the addingunit, and when the number of pixels counted by said counting unit is apower of two, said adding unit determines a mean luminance value.
 87. Animage binarization apparatus comprising: a block division unit whichdivides a multi-valued image into blocks; a luminance value output unitwhich outputs a luminance value of each pixel forming the multi-valuedimage; binarization threshold value setting unit which sets abinarization threshold value to be used when binarizing the multi-valuedimage; a mean luminance value calculation unit which receives luminancevalues of each pixel forming at least selected of the blocks from amongluminance values output by said luminance value output unit, andcalculates mean luminance values of the selected of the blocks; aluminance value limiting unit which limits a range of the mean luminancevalues calculated by said mean luminance value calculation unit so thatthe values are within a predetermined spread; a block binarizationthreshold value setting unit which sets a block binarization thresholdvalue which is a binarization threshold value applied to the selected ofthe blocks based on the mean luminance values calculated by said meanluminance value calculation unit; and an interpolation block settingunit which sets interpolation blocks which cover pixels extending overtwo or more adjacent of the selected of the blocks, wherein saidbinarization threshold value setting unit sets binarization thresholdvalues applied to pixels inside the interpolation block based on blockbinarization threshold values of each of the two or more adjacent of theselected of the blocks bridged by the interpolation block.
 88. The imagebinarization apparatus according to claim 87, wherein said blockdivision unit changes the size of created blocks in accordance with theimage size of the multi-valued image or with the total number of pixelsof the multi-valued image.
 89. The image binarization apparatusaccording to claim 87, wherein said block division unit changes the sizeor shape of blocks to be created in accordance with the positions ofblocks to be created within the multi-valued image.
 90. The imagebinarization apparatus according to claim 87 further comprises asampling unit which samples pixels which form the multi-valued image,wherein said luminance value output unit outputs luminance values ofpixels sampled by said sampling unit.
 91. The image binarizationapparatus according to claim 90, wherein said sampling unit sets asampling interval used in the sampling in accordance with the imagesize, the total number of pixels, or the block size.
 92. The imagebinarization apparatus according to claim 87, wherein said meanluminance value calculation unit comprises an adding unit which adds theluminance values of each pixel; and a counting unit which counts thenumber of pixels added by the adding unit, and when the number of pixelscounted by said counting unit is a power of two, said adding unitdetermines a mean luminance value.
 93. An image pickup apparatuscomprising: an image pickup unit which picks up an image of an object ofa photograph; a screen division unit which divides the photographedobject into a plurality of screens; a photometry unit which measureslight of screens created by said screen division unit; a block divisionunit which divides a multi-valued image picked up by said image pickupunit into blocks; a binarization threshold value setting unit which setsbinarization threshold values used when binarizing the multi-valuedimage; a photometric value smoothing unit which smoothes the photometricvalues measured by said photometry unit; and an interpolation blocksetting unit which sets interpolation blocks which cover an image areaextending over two or more adjacent screens from among the screenscreated by said screen division unit, wherein said binarizationthreshold value setting unit sets binarization threshold values appliedto pixels of the interpolation blocks based on smoothed photometricvalues of each of the two or more screens bridged by the interpolationblocks.
 94. The image pickup apparatus according to claim 93, whereinscreens created by said screen division unit are identical to blockscreated by said block division unit.
 95. An image pickup apparatuscomprising: an image pickup unit which picks up an image of an object ofa photograph; a screen division unit which divides the photographedobject into a plurality of screens; a photometry unit which measureslight of screens created by said screen division unit; a block divisionunit which divides a multi-valued image picked up by said image pickupunit into blocks; a binarization threshold value setting unit which setsbinarization threshold values used when binarizing the multi-valuedimage; and a photometric value limiting unit which limits a spread ofphotometric values measured by said photometry unit so that the valuesare within a predetermined range, wherein said binarization thresholdvalue setting unit sets binarization threshold values of blocks createdby said block division unit based on photometric values the range ofwhich has been limited by said photometric value limiting unit.
 96. Theimage pickup apparatus according to claim 95, wherein screens created bysaid screen division unit are identical to blocks created by said blockdivision unit.
 97. An image pickup apparatus comprising: an image pickupunit which picks up an image of an object of a photograph; a screendivision unit which divides the object being photographed into aplurality of screens; a photometry unit which measures light of screenscreated by said screen division unit; a block division unit whichdivides a multi-valued image picked up by said image pickup unit intoblocks; a binarization threshold value setting unit which setsbinarization threshold values used when binarizing the multi-valuedimage; a photometric value limiting unit which limits a spread ofphotometric values measured by said photometry unit so that the valuesare within a predetermined range; and an interpolation block settingunit which sets interpolation blocks which cover an image area extendingover two or more adjacent screens from among the screens created by saidscreen division unit, wherein said binarization threshold value settingunit sets binarization threshold values to be applied to pixels withinthe interpolation blocks based on photometric values the range of eachof which has been limited of the two or more screens bridged by theinterpolation block.
 98. The image pickup apparatus according to claim97, wherein screens created by said screen division unit are identicalto blocks created by said block division unit.
 99. An image binarizationmethod for performing binarization processing on a multi-valued imagecomprising: a block division step in which the multi-valued image isdivided into blocks; a low luminance threshold value setting step inwhich a low luminance threshold value to be used when removing lowluminance values is set based on mean luminance values of adjacent of atleast selected of the blocks; a low luminance value removal step inwhich luminance values below the low luminance threshold value areremoved from among luminance values of pixels contained in the selectedof the blocks and only luminance values which exceed the low luminancethreshold value are output; a mean luminance value calculation step inwhich luminance values output in the low luminance value removal stepare input and mean luminance values of the selected of the blocks arecalculated; and a binarization threshold value setting step in whichbinarization threshold values to be used in binarization processing ofthe object blocks are set based on mean luminance values of the blockscalculated in the mean luminance value calculation step.
 100. The imagebinarization method according to claim 99, wherein, in the blockdivision step, the size of a block is changed in accordance with theimage size of the multi-valued image or with the total number of pixelsof the multi-valued image.
 101. The image binarization method accordingto claim 99, wherein, in the block division step, the size or shape ofblocks to be created is changed in accordance with the positions ofblocks to be created within the multi-valued image.
 102. The imagebinarization method according to claim 101 further comprising a samplingstep for sampling pixels which form the multi-valued image, and in thelow luminance value removal step, using pixels sampled in the samplingstep, luminance values below the low luminance threshold value areremoved from luminance values of the pixels and only luminance valueswhich exceed the threshold luminance value are output.
 103. The imagebinarization method according to claim 102, wherein, in the samplingstep, a sampling interval used in the sampling is set in accordance withthe image size, the total number of pixels, or the block size.
 104. Animage binarization method for performing binarization processing on amulti-valued image comprising: a block division step in which themulti-valued image is divided into blocks; a low luminance thresholdvalue setting step in which a low luminance threshold value to be usedwhen removing low luminance values is set based on mean luminance valuesof adjacent of at least selected of the blocks; a low luminance valueremoval step in which luminance values below the low luminance thresholdvalue are removed from among luminance values of pixels contained in theselected of the blocks and only luminance values which exceed the lowluminance threshold value are output; a mean luminance value calculationstep in which luminance values output in the low luminance value removalstep are input and mean luminance values of the selected of the blocksare calculated; a block binarization threshold value setting step inwhich a block binarization threshold value which is a binarizationthreshold value applied to a block is set based on a mean luminancevalue calculated in the mean luminance value calculation step; aninterpolation block setting step in which interpolation blocks whichcover pixels extending over two or more adjacent of the selected of theblocks are set; and a binarization threshold value setting step in whichbinarization threshold values to be applied to pixels within theinterpolation blocks are set based on each block binarization thresholdvalue of the two or more adjacent of the selected of the blocks bridgedby the interpolation block set in the interpolation block setting step.105. The image binarization method according to claim 104, wherein, inthe block division step, the size of a block is changed in accordancewith the image size of the multi-valued image or with the total numberof pixels of the multi-valued image.
 106. The image binarization methodaccording to claim 104, wherein, in the block division step, the size orshape of blocks to be created is changed in accordance with thepositions of blocks to be created within the multi-valued image. 107.The image binarization method according to claim 104 further comprisinga sampling step for sampling pixels which form the multi-valued image,and in the low luminance value removal step, using pixels sampled in thesampling step, luminance values below the low luminance threshold valueare removed from luminance values of the pixels and only luminancevalues which exceed the threshold luminance value are output.
 108. Theimage binarization method according to claim 107, wherein, in thesampling step, a sampling interval used in the sampling is set inaccordance with the image size, the total number of pixels, or the blocksize.
 109. An image binarization method for performing binarizationprocessing on a multi-valued image comprising: a block division step inwhich the multi-valued image is divided into blocks; a mean luminancevalue calculation step in which mean luminance values of at leastselected of the blocks are calculated; a luminance value limiting stepin which a spread of mean luminance values calculated in the meanluminance value calculation step is limited so that the values arewithin a predetermined range; and a binarization threshold value settingstep in which a binarization threshold value to be used in binarizationprocessing of the selected of the blocks is set based on mean luminancevalues the range of which has been limited in the luminance valuelimiting step.
 110. The image binarization method according to claim109, wherein, in the block division step, the size of a block is changedin accordance with the image size of the multi-valued image or with thetotal number of pixels of the multi-valued image.
 111. The imagebinarization method according to claim 109, wherein, in the blockdivision step, the size or shape of blocks to be created is changed inaccordance with the positions of blocks to be created within themulti-valued image.
 112. The image binarization method according toclaim 109, wherein the image binarization method further comprises asampling step for sampling pixels which form the multi-valued image, andin the mean luminance value calculation step, mean luminance values arecalculated using pixels sampled in the sampling step.
 113. The imagebinarization method according to claim 112, wherein, in the samplingstep, a sampling interval used in the sampling is set in accordance withthe image size, the total number of pixels, or the block size.
 114. Animage binarization method for performing binarization processing on amulti-valued image comprising: a block division step in which themulti-valued image is divided into blocks; a mean luminance valuecalculation step in which mean luminance values of at least selected ofthe blocks are calculated; a luminance value limiting step in which aspread of mean luminance values calculated in the mean luminance valuecalculation step is limited so that the values are within apredetermined range; a block binarization threshold value setting stepin which a block binarization threshold value which is a binarizationthreshold value applied to the selected of the blocks is set based onmean luminance values the range of which has been limited in the meanluminance value limiting step; an interpolation block setting step inwhich interpolation blocks which share pixels extending over two or moreadjacent of the selected of the blocks are set; and a binarizationthreshold value setting step in which binarization threshold values tobe applied to pixels within the interpolation blocks are set based oneach block binarization threshold value of the two or more adjacent ofthe selected of the blocks bridged by the interpolation block set in theinterpolation block setting step.
 115. The image binarization methodaccording to claim 114, wherein, in the block division step, the size ofa block is changed in accordance with the image size of the multi-valuedimage or with the total number of pixels of the multi-valued image. 116.The image binarization method according to claim 114, wherein, in theblock division step, the size or shape of blocks to be created ischanged in accordance with the positions of blocks to be created withinthe multi-valued image.
 117. The image binarization method according toclaim 114, wherein the image binarization method further comprises asampling step for sampling pixels which form the multi-valued image, andin the mean luminance value calculation step, mean luminance values arecalculated using pixels sampled in the sampling step.
 118. The imagebinarization method according to claim 114, wherein, in the samplingstep, a sampling interval used in the sampling is set in accordance withthe image size, the total number of pixels, or the block size.
 119. Animage pickup method for performing binarization processing on amulti-valued image comprising: a screen division step in which an objectof a photograph is divided into a plurality of screens; a photometrystep in which light of screens created in the screen division step ismeasured; an image pickup step in which an image of the object of thephotograph is picked up; a block division step in which a multi-valuedimage which was picked up in the image pickup step is divided intoblocks; a photometric value smoothing step in which photometric valuesmeasured in the photometry step are smoothed; an interpolation blocksetting step in which interpolation blocks which cover an image areaextending over two or more adjacent screens are set from among thescreens created in the screen division step; and a binarizationthreshold value setting step in which binarization threshold valuesapplied to pixels within the interpolation blocks are set based onsmoothed photometric values of each of the two or more screens bridgedby the interpolation blocks set in the interpolation block setting step.120. The image pickup method according to claim 119, wherein screenscreated in the screen division step are identical to blocks created inthe block division step.
 121. An image pickup method for performingbinarization processing on a multi-valued image comprising: a screendivision step in which an object of a photograph is divided into aplurality of screens; a photometry step in which light of screenscreated in the screen division step is measured; an image pickup step inwhich an image of the object of the photograph is picked up; a blockdivision step in which a multi-valued image which was picked up in theimage pickup step is divided into blocks; a photometric value limitingstep in which the spread of photometric values measured in thephotometry step is limited so that the values are within a predeterminedrange; and a binarization threshold value setting step in whichbinarization threshold values of at least selected of the blocks are setbased on photometric values whose range has been limited in thephotometric value limiting step.
 122. The image pickup method accordingto claim 121, wherein screens created in the screen division step areidentical to blocks created in the block division step.
 123. An imagepickup method for performing binarization processing on a multi-valuedimage comprising: a screen division step in which an object of aphotograph is divided into a plurality of screens; a photometry step inwhich light of screens created in the screen division step is measured;an image pickup step in which an image of the object of the photographis picked up; a block division step in which a multi-valued image whichwas picked up in the image pickup step is divided into blocks; aphotometric value limiting step in which the spread of photometricvalues measured in the photometry step is limited so that the values arewithin a predetermined range; an interpolation block setting step inwhich interpolation blocks which cover an image area extending over twoor more adjacent screens are set from among the screens created in thescreen division step; and a binarization threshold value setting step inwhich binarization threshold values applied to pixels within theinterpolation blocks are set based on photometric values the range ofeach of which has been limited of the two or more screens bridged by theinterpolation blocks set in the interpolation block setting step. 124.The image pickup method according to claim 123, wherein screens createdin the screen division step are identical to blocks created in the blockdivision step.
 125. A computer readable medium for storing instructions,which when executed by a computer, causes the computer to perform animage binarization method comprising: a block division step in which themulti-valued image is divided into blocks; a low luminance thresholdvalue setting step in which a low luminance threshold value to be usedwhen removing low luminance values is set based on mean luminance valuesof adjacent of at least selected of the blocks; a low luminance valueremoval step in which luminance values below the low luminance thresholdvalue are removed from among luminance values of pixels contained in theselected of the blocks and only luminance values which exceed the lowluminance threshold value are output; a mean luminance value calculationstep in which luminance values output in the low luminance value removalstep are input and mean luminance values of the selected of the blocksare calculated; and a binarization threshold value setting step in whichbinarization threshold values to be used in binarization processing ofthe selected of the blocks are set based on mean luminance values of theselected of the blocks calculated in the mean luminance valuecalculation step.
 126. A computer readable medium for storinginstructions, which when executed by a computer, causes the computer toperform an image binarization method comprising: a block division stepin which the multi-valued image is divided into blocks; a low luminancethreshold value setting step in which a low luminance threshold value tobe used when removing low luminance values is set based on meanluminance values of adjacent of at least selected of the blocks; a lowluminance value removal step in which luminance values below the lowluminance threshold value are removed from among luminance values ofpixels contained in the selected of the blocks and only luminance valueswhich exceed the low luminance threshold value are output; a meanluminance value calculation step in which luminance values output in thelow luminance value removal step are input and mean luminance values ofthe selected of the blocks are calculated; a block binarizationthreshold value setting step in which a block binarization thresholdvalue which is a binarization threshold value applied to a block is setbased on a mean luminance value calculated in the mean luminance valuecalculation step; an interpolation block setting step in whichinterpolation blocks which cover pixels extending over two or moreadjacent of the selected of the blocks are set; and a binarizationthreshold value setting step in which binarization threshold values tobe applied to pixels within the interpolation blocks are set based oneach block binarization threshold value of the two or more adjacent ofthe selected blocks bridged by the interpolation block set in theinterpolation block setting step.
 127. A computer readable medium forstoring instructions, which when executed by a computer, causes thecomputer to perform an image binarization method comprising: a blockdivision step in which the multi-valued image is divided into blocks; amean luminance value calculation step in which mean luminance values ofat least selected of the blocks are calculated; a luminance valuelimiting step in which a spread of mean luminance values calculated inthe mean luminance value calculation step is limited so that the valuesare within a predetermined range; and a binarization threshold valuesetting step in which binarization threshold value to be used inbinarization processing of the selected of the blocks is set based onmean luminance values the range of which has been limited in theluminance value limiting step.
 128. A computer readable medium forstoring instructions, which when executed by a computer, causes thecomputer to perform an image binarization method comprising: a blockdivision step in which the multi-valued image is divided into blocks; amean luminance value calculation step in which mean luminance values ofat least selected of the blocks are calculated; a luminance valuelimiting step in which a spread of mean luminance values calculated inthe mean luminance value calculation step is limited so that the valuesare within a predetermined range; a block binarization threshold valuesetting step in which a block binarization threshold value which is abinarization threshold value applied to the at least selected of theblocks is set based on mean luminance values the range of which has beenlimited in the mean luminance value limiting step; an interpolationblock setting step in which interpolation blocks which share pixelsextending over two or more adjacent of the selected of the blocks areset; and a binarization threshold value setting step in whichbinarization threshold values to be applied to pixels within theinterpolation blocks are set based on each block binarization thresholdvalue of the two or more adjacent of the selected of the blocks bridgedby the interpolation block set in the interpolation block setting step.129. A computer readable medium for storing instructions, which whenexecuted by a computer, causes the computer to perform an image pickupmethod comprising: a screen division step in which an object of aphotograph is divided into a plurality of screens; a photometry step inwhich light of screens created in the screen division step is measured;an image pickup step in which an image of the object of the photographis picked up; a block division step in which a multi-valued image whichwas picked up in the image pickup step is divided into blocks; aphotometric value smoothing step in which photometric values measured inthe photometry step are smoothed; and a binarization threshold valuesetting step in which binarization threshold values of at least selectedof the blocks are set based on photometric values smoothed in thephotometric value smoothing step.
 130. A computer readable medium forstoring instructions, which when executed by a computer, causes thecomputer to perform an image pickup method comprising: a screen divisionstep in which an object of a photograph is divided into a plurality ofscreens; a photometry step in which light of screens created in thescreen division step is measured; an image pickup step in which an imageof the object of the photograph is picked up; a block division step inwhich a multi-valued image which was picked up in the image pickup stepis divided into blocks; a photometric value smoothing step in whichphotometric values measured in the photometry step are smoothed; aninterpolation block setting step in which interpolation blocks whichcover an image area extending over two or more adjacent screens are setfrom among the screens created in the screen division step; and abinarization threshold value setting step in which binarizationthreshold values applied to pixels within the interpolation blocks areset based on smoothed photometric values of each of the two or morescreens bridged by the interpolation blocks set in the interpolationblock setting step.
 131. A computer readable medium for storinginstructions, which when executed by a computer, causes the computer toperform an image pickup method comprising: a screen division step inwhich an object of a photograph is divided into a plurality of screens;a photometry step in which light of screens created in the screendivision step is measured; an image pickup step in which an image of theobject of the photograph is picked up; a block division step in which amulti-valued image which was picked up in the image pickup step isdivided into blocks; a photometric value limiting step in which thespread of photometric values measured in the photometry step is limitedso that the values are within a predetermined range; and a binarizationthreshold value setting step in which binarization threshold values ofthe at least selected of blocks are set based on photometric valueswhose range has been limited in the photometric value limiting step.132. computer readable medium for storing instructions, which whenexecuted by a computer, causes the computer to perform an image pickupmethod comprising: a screen division step in which an object of aphotograph is divided into a plurality of screens; a photometry step inwhich light of screens created in the screen division step is measured;an image pickup step in which an image of the object of the photographis picked up; a block division step in which a multi-valued image whichwas picked up in the image pickup step is divided into blocks; aphotometric value limiting step in which the spread of photometricvalues measured in the photometry step is limited so that the values arewithin a predetermined range; an interpolation block setting step inwhich interpolation blocks which cover an image area extending over twoor more adjacent screens are set from among the screens created in thescreen division step; and a binarization threshold value setting step inwhich binarization threshold values applied to pixels within theinterpolation blocks are set based on photometric values the range ofeach of which has been limited of the two or more screens bridged by theinterpolation blocks set in the interpolation block setting step.